-- Logs begin at Wed 2022-08-24 08:00:03 UTC, end at Wed 2022-08-24 09:18:08 UTC. -- Aug 24 08:32:47.082828 dv-cinder-852210-4-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 24 08:32:47.091601 dv-cinder-852210-4-iscsi devstack@keystone.service[93532]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 24 08:32:47.093807 dv-cinder-852210-4-iscsi devstack@keystone.service[93532]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Aug 24 08:32:47 2022] *** Aug 24 08:32:47.093807 dv-cinder-852210-4-iscsi devstack@keystone.service[93532]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 24 08:32:47.093807 dv-cinder-852210-4-iscsi devstack@keystone.service[93532]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 24 08:32:47.093807 dv-cinder-852210-4-iscsi devstack@keystone.service[93532]: nodename: dv-cinder-852210-4-iscsi Aug 24 08:32:47.093807 dv-cinder-852210-4-iscsi devstack@keystone.service[93532]: machine: x86_64 Aug 24 08:32:47.093807 dv-cinder-852210-4-iscsi devstack@keystone.service[93532]: clock source: unix Aug 24 08:32:47.093807 dv-cinder-852210-4-iscsi devstack@keystone.service[93532]: pcre jit disabled Aug 24 08:32:47.093807 dv-cinder-852210-4-iscsi devstack@keystone.service[93532]: detected number of CPU cores: 4 Aug 24 08:32:47.093807 dv-cinder-852210-4-iscsi devstack@keystone.service[93532]: current working directory: / Aug 24 08:32:47.093807 dv-cinder-852210-4-iscsi devstack@keystone.service[93532]: detected binary path: /usr/bin/uwsgi-core Aug 24 08:32:47.093807 dv-cinder-852210-4-iscsi devstack@keystone.service[93532]: your processes number limit is 47680 Aug 24 08:32:47.094520 dv-cinder-852210-4-iscsi devstack@keystone.service[93532]: your memory page size is 4096 bytes Aug 24 08:32:47.094520 dv-cinder-852210-4-iscsi devstack@keystone.service[93532]: detected max file descriptor number: 2048 Aug 24 08:32:47.094520 dv-cinder-852210-4-iscsi devstack@keystone.service[93532]: lock engine: pthread robust mutexes Aug 24 08:32:47.094520 dv-cinder-852210-4-iscsi devstack@keystone.service[93532]: thunder lock: enabled Aug 24 08:32:47.094520 dv-cinder-852210-4-iscsi devstack@keystone.service[93532]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 24 08:32:47.094520 dv-cinder-852210-4-iscsi devstack@keystone.service[93532]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 24 08:32:47.134062 dv-cinder-852210-4-iscsi devstack@keystone.service[93532]: Python main interpreter initialized at 0x55e4d921d9c0 Aug 24 08:32:47.134062 dv-cinder-852210-4-iscsi devstack@keystone.service[93532]: python threads support enabled Aug 24 08:32:47.134062 dv-cinder-852210-4-iscsi devstack@keystone.service[93532]: your server socket listen backlog is limited to 100 connections Aug 24 08:32:47.134062 dv-cinder-852210-4-iscsi devstack@keystone.service[93532]: your mercy for graceful operations on workers is 90 seconds Aug 24 08:32:47.141695 dv-cinder-852210-4-iscsi devstack@keystone.service[93532]: mapped 403077 bytes (393 KB) for 2 cores Aug 24 08:32:47.141695 dv-cinder-852210-4-iscsi devstack@keystone.service[93532]: *** Operational MODE: preforking *** Aug 24 08:32:47.141695 dv-cinder-852210-4-iscsi devstack@keystone.service[93532]: *** uWSGI is running in multiple interpreter mode *** Aug 24 08:32:47.141695 dv-cinder-852210-4-iscsi devstack@keystone.service[93532]: spawned uWSGI master process (pid: 93532) Aug 24 08:32:47.141695 dv-cinder-852210-4-iscsi devstack@keystone.service[93532]: spawned uWSGI worker 1 (pid: 93542, cores: 1) Aug 24 08:32:47.141695 dv-cinder-852210-4-iscsi devstack@keystone.service[93532]: spawned uWSGI worker 2 (pid: 93543, cores: 1) Aug 24 08:32:47.141695 dv-cinder-852210-4-iscsi devstack@keystone.service[93532]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 24 08:32:47.135083 dv-cinder-852210-4-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 24 08:32:51.425191 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 08:32:51.425191 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: from cryptography.utils import int_from_bytes Aug 24 08:32:51.426497 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 08:32:51.426497 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: from cryptography.utils import int_from_bytes Aug 24 08:32:51.426929 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 08:32:51.426929 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: from cryptography.utils import int_from_bytes Aug 24 08:32:51.427401 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 08:32:51.427401 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: from cryptography.utils import int_from_bytes Aug 24 08:32:51.484164 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 24 08:32:51.484164 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 24 08:32:51.484164 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] command line args: [] {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 24 08:32:51.484164 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 24 08:32:51.484164 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] ================================================================================ {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 24 08:32:51.484164 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] admin_token = **** {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:32:51.484164 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] config_dir = [] {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:32:51.484164 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:32:51.484765 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 24 08:32:51.484842 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] config_source = [] {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:32:51.484908 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 24 08:32:51.484976 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] debug = True {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:32:51.485095 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] command line args: [] {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 24 08:32:51.485183 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:32:51.485528 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 24 08:32:51.485596 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:32:51.485660 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] ================================================================================ {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 24 08:32:51.485725 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] insecure_debug = False {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:32:51.485790 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] admin_token = **** {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:32:51.485854 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:32:51.485920 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] config_dir = [] {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:32:51.485980 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:32:51.486037 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:32:51.486101 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] list_limit = None {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:32:51.486168 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] config_source = [] {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:32:51.486233 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] log_config_append = None {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:32:51.486301 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] debug = True {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:32:51.486367 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:32:51.486432 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:32:51.486519 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] log_dir = None {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:32:51.486581 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:32:51.486649 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] log_file = None {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:32:51.486710 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] insecure_debug = False {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:32:51.486767 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:32:51.486830 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:32:51.486894 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:32:51.486953 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:32:51.487021 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:32:51.487081 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] list_limit = None {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:32:51.487144 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:32:51.487211 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] log_config_append = None {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:32:51.487269 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:32:51.487333 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:32:51.487403 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:32:51.487464 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] log_dir = None {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:32:51.487464 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] log_file = None {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:32:51.487571 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:32:51.487635 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:32:51.487716 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:32:51.487716 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:32:51.487820 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:32:51.487908 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:32:51.487982 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:32:51.488047 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:32:51.488114 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:32:51.488114 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:32:51.488225 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:32:51.488296 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:32:51.488362 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:32:51.488949 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:32:51.489023 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] notification_format = cadf {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:32:51.489084 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:32:51.489148 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:32:51.489213 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:32:51.489213 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:32:51.489334 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.55/identity {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:32:51.489400 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:32:51.489459 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] publish_errors = False {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:32:51.489517 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:32:51.489573 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:32:51.489638 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:32:51.489697 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:32:51.489697 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:32:51.489797 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] notification_format = cadf {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:32:51.489859 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:32:51.489979 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:32:51.490043 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:32:51.490043 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] standard_threads = False {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:32:51.490152 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.55/identity {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:32:51.490217 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] strict_password_check = False {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:32:51.490315 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] publish_errors = False {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:32:51.490400 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:32:51.490465 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:32:51.490525 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] use_eventlog = False {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:32:51.490592 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:32:51.490657 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] use_journal = False {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:32:51.490716 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:32:51.490779 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] use_json = False {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:32:51.490779 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] use_stderr = False {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:32:51.490892 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:32:51.490956 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] use_syslog = False {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:32:51.491021 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:32:51.491080 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] watch_log_file = False {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:32:51.491142 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] standard_threads = False {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:32:51.491201 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.491258 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] strict_password_check = False {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:32:51.491322 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.491382 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:32:51.491696 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.491766 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] use_eventlog = False {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:32:51.491766 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] use_journal = False {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:32:51.491875 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.492011 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] use_json = False {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:32:51.492073 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.492130 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] use_stderr = False {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:32:51.492187 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.492244 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] use_syslog = False {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:32:51.492307 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.492373 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] watch_log_file = False {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:32:51.492432 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.492495 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.492589 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.492670 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.492736 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.492803 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.492864 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.492921 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.492979 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.493035 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.493098 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.493165 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.493225 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.493283 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.493340 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.493403 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.493467 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.493534 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.493594 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.493652 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.493725 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.493785 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.493854 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.493914 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.493972 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.494029 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.494092 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.494151 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.494215 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.494275 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.494333 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.494391 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.494454 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.494525 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.494595 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.494667 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.494726 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.494783 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.494840 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.494897 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.495028 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.495091 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.495149 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.495207 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.495270 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.495329 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.495398 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.495398 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.495507 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.495571 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.495630 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.495695 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.495760 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.495820 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.495877 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.495934 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.495991 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.496053 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.496141 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.496213 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.496273 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.496336 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.496395 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.496453 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.496518 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.497455 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.497524 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.497583 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.497653 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.497713 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.497780 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.497846 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.497905 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.497962 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.498059 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.498129 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.498240 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.498356 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.498453 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.498556 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.498653 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.498766 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.498870 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.498973 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.499086 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.499194 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.499296 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.499401 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.499535 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.499623 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.499707 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.499781 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.499858 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.499931 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.500002 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.500080 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.500159 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.500238 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.500318 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.500391 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.500463 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.500570 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.500693 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.500806 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.500944 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.501043 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.501178 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.501280 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.501415 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.501518 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.501652 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.501759 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.501888 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.501996 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.502149 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.502272 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.502387 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.502511 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.502618 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.502747 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.502910 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.503016 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.503151 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.503252 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.503386 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.503500 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.503617 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.503745 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.503877 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.503980 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.504692 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.504796 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.504913 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.505000 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.505135 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.505222 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] cache.enabled = True {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.505371 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.505530 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.505641 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.505721 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.505880 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.505962 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.506130 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.506217 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.506368 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.506448 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.506602 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.506680 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.506835 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.506927 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.507069 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.507150 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.507317 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] cache.enabled = True {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.507397 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.507555 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.507635 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.507793 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.507879 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.508033 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.508112 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.508275 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.508353 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.508513 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.508615 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.508772 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.508852 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.509006 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.509083 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.509238 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.509320 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.509518 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.509597 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.509753 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.509831 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.509991 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.510070 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.510221 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.510305 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.510460 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.510539 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.510695 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.510775 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.510933 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.511011 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.511167 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.511244 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.511398 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.511477 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.511633 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.511712 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.511867 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.511945 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.512104 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.512184 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] auth.external = None {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.512333 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] auth.mapped = None {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.512416 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.512596 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.512679 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.512839 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.512920 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.513091 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.513189 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] auth.password = None {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.513462 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] auth.token = None {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.513544 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.513618 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.513694 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.513793 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] catalog.caching = True {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.513881 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.514003 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.514085 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.514232 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.514318 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] auth.external = None {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.514480 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.514561 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] auth.mapped = None {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.514724 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.514803 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.514954 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.515034 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.515188 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] credential.caching = True {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.515322 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] auth.password = None {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.515432 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] credential.driver = sql {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.515560 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] auth.token = None {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.515680 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.515806 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.515927 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.516045 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] catalog.caching = True {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.516160 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.516296 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.516403 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.516528 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.516649 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.516803 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.516884 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.517051 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.517131 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.517287 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.517376 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.517523 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] credential.caching = True {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.517607 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.517772 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] credential.driver = sql {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.517852 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.518023 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.518105 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.518278 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.518365 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.518516 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.518603 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.518763 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.518846 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.519001 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.519086 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] federation.caching = True {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.519254 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.519335 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.519501 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] federation.driver = sql {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.519656 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.519763 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.519850 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.519992 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.520071 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.520239 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.520321 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.520473 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.520623 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.520752 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.530043 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.530177 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.530262 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.530336 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.530410 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.530483 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.530555 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] federation.caching = True {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.530636 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.530711 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.530784 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] identity.caching = True {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.530857 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] federation.driver = sql {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.530930 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.531003 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.531083 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.531158 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.531230 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.531303 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.531375 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.531447 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.531527 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] identity.driver = sql {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.531602 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.531682 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.531756 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.531828 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.531901 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.531982 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.532056 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.532128 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.532200 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.532273 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.532346 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] identity.caching = True {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.532426 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.532500 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.532591 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.532665 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.532738 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.532810 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.532895 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.532971 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.533044 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.533116 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] identity.driver = sql {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.533188 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.533261 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.533341 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.533416 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.533488 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.533560 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.533632 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.533704 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.533785 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.533860 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.533932 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.534005 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.534077 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.534171 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.534254 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.534327 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.534400 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.534474 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.534606 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.534694 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.534770 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.534770 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.534900 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.534973 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.535044 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.535124 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.535198 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.535271 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.535342 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.535414 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.535485 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.535565 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.535639 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.535711 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.535783 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.535854 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.535926 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.536006 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.536080 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.536164 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] ldap.password = **** {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.536239 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.536312 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.536389 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.536471 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.536581 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.536581 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.536718 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.536718 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.536848 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.536933 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.537009 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.537082 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.537155 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.537228 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.537300 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.537380 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.537455 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.537527 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.537589 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.537661 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.537740 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.537822 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.537896 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] ldap.password = **** {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.537969 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.538040 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.538112 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.538184 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.538313 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.538396 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.538468 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] ldap.user = None {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.538540 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.538540 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.538678 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.538757 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.538832 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.538903 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.538976 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.539053 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.539126 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.539207 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.539207 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.539338 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.539410 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.539481 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.539552 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.539552 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.539693 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.539754 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.539814 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.539884 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] ldap.user = None {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.539884 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.540022 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.540097 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.540169 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.540230 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.540230 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.540349 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.540417 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.540479 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.540573 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.540650 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.540723 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.540795 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.540879 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.540955 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.541028 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.541100 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.541173 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.541246 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.541325 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.541400 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.541473 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.541546 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.541618 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.541691 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.541771 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.541845 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.541917 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.541989 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.542061 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.542134 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.542213 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.542288 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.542360 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.542439 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.542513 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.542593 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.542668 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.542740 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] policy.driver = sql {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.542812 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.542884 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.542956 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.543035 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.543110 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.543182 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.543243 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.543303 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] receipt.caching = True {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.543374 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.543522 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.543598 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.543671 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.543743 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] policy.driver = sql {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.543816 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.543888 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.543968 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.544043 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.544115 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.544187 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.544253 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] resource.caching = True {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.544324 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] receipt.caching = True {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.544405 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.544480 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.544568 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] resource.driver = sql {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.544633 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.544705 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.544778 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.544866 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.544943 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.545016 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.545088 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.545160 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] revoke.caching = True {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.545233 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] resource.caching = True {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.545313 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.545387 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.545460 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.545532 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] resource.driver = sql {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.545605 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] role.cache_time = None {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.545716 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.545840 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] role.caching = True {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.545969 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.546086 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] role.driver = sql {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.546222 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.546303 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] role.list_limit = None {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.546458 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] revoke.caching = True {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.546552 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.546708 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.546791 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.546946 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.547026 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.547188 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] role.cache_time = None {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.547283 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.547415 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] role.caching = True {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.547494 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.547663 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] role.driver = sql {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.547744 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.547893 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] role.list_limit = None {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.547982 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.548154 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.548234 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.548387 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.548466 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.548662 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.548759 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.548910 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.548992 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.549156 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.549236 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.549402 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.549489 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.549650 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.549816 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.549901 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.550053 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.550133 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.550304 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.550386 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.550551 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.550631 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.550798 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.551406 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.551563 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.551644 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.551797 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.551984 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.552065 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.552225 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.552306 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.552478 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.552591 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.552751 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.552842 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.552985 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.553067 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.553216 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.553296 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.553465 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.553545 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.553721 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.553810 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.554002 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.554096 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.554203 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.554283 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.554437 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.554525 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.554666 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] token.cache_time = None {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.554746 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.554898 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] token.caching = True {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.554978 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.555131 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.555217 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.555377 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] token.provider = fernet {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.555457 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.555611 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.555690 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.555853 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.555934 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.556094 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.556175 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.556325 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] token.cache_time = None {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.556405 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.556577 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] token.caching = True {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.556663 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.556817 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.556897 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.557062 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] trust.driver = sql {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.557141 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] token.provider = fernet {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.557296 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.557378 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.557529 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.557606 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.557759 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.557839 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.558000 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.558082 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.558245 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.558324 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.558468 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.558546 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.558709 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.558851 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] trust.driver = sql {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.559003 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.559174 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.559258 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.559331 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] database.connection = **** {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.559471 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.559566 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.559719 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.559801 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.559951 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.560030 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.560188 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.560278 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.560417 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.560504 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.560674 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.560756 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.560908 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.561005 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] database.connection = **** {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.561142 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.561233 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.561384 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.561465 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.561616 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.561696 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.561842 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.561919 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.562075 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.562156 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.562320 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.562402 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.562554 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.562636 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.562793 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.562874 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.563022 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.563102 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.563257 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.563339 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.563489 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.563569 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.563642 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=93543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 24 08:32:51.563715 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.563957 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.564198 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.564440 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.564697 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.564951 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.565194 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:32:51.565365 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=93542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 24 08:32:51.593280 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=93543) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:32:51.593527 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=93543) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:32:51.593729 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=93543) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:32:51.594150 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=93542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:32:51.594383 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=93542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:32:51.594608 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=93542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:32:51.602184 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=93543) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:32:51.603095 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=93542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:32:51.631131 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=93543) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:32:51.631352 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=93543) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:32:51.632766 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=93542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:32:51.633001 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=93542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:32:51.685308 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=93543) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:32:51.686008 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=93542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:32:51.739112 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=93543) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:32:51.739447 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=93543) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:32:51.739755 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=93542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:32:51.739992 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=93542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:32:51.896680 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93543) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:32:51.896974 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=93543) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:32:51.897092 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:32:51.897181 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=93543) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:32:51.897259 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=93542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:32:51.897336 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=93543) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:32:51.897481 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=93542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:32:51.897578 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=93543) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:32:51.897704 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=93542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:32:51.897789 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=93543) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:32:51.897923 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=93542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:32:51.898161 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=93542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:32:51.929657 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:32:51.929861 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93543) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:32:51.929957 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=93542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:32:51.930035 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=93543) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:32:51.930110 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=93542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:32:51.930182 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=93543) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:32:51.930305 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=93542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:32:51.930391 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=93543) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:32:51.930518 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=93543) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:32:51.930603 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=93542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:32:51.930728 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=93543) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:32:51.930821 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=93542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:32:51.930938 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=93543) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:32:51.931091 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=93542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:32:51.942145 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=93543) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:32:51.942543 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=93542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:32:51.961440 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93543) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:32:51.961673 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=93543) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:32:51.961761 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:32:51.961898 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=93543) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:32:51.962028 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=93542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:32:51.962112 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=93543) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:32:51.962187 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=93542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:32:51.962323 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=93543) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:32:51.962408 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=93542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:32:51.962534 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=93543) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:32:51.962618 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=93542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:32:51.962745 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=93543) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:32:51.962831 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=93542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:32:51.962981 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=93542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:32:51.979846 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=93543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:51.980054 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=93543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 08:32:51.980397 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=93543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:51.980496 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=93542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:51.980625 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=93543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 08:32:51.980710 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=93542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 08:32:51.980902 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=93543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:51.980995 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=93542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:51.981162 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=93543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:51.981248 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=93542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 08:32:51.981456 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=93542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:51.981668 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=93543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:51.981754 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=93542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:51.981932 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=93543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:51.982268 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=93542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:51.982535 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=93542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:51.994187 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=93543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:51.994187 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=93543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:51.994187 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=93543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:51.994451 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=93542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:51.994551 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=93543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:51.994633 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=93542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:51.994633 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=93542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:51.994633 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=93542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:51.999869 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=93543) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:32:52.000722 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=93542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:32:52.002779 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=93543) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:32:52.003060 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=93543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.003340 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=93543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.003601 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=93542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:32:52.003700 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=93543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.003888 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=93542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.003972 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=93543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.004174 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=93543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.004260 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=93542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.004449 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=93543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.004560 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=93542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.004752 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=93543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.004865 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=93542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.005044 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=93542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.005170 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=93543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.005320 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=93542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.005539 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=93543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.005632 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=93542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.005818 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=93543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.006032 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=93542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.006319 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=93542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.006594 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=93542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.028333 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=93543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.029542 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=93542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.029830 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=93543) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:32:52.034456 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=93543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.034628 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=93542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:32:52.034628 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=93542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.034794 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=93543) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:32:52.035075 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=93543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.035877 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=93543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.036565 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=93542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:32:52.036848 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=93542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.037131 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=93542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.042226 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=93543) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:32:52.042507 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=93543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.043488 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=93542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:32:52.043776 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=93542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.046584 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=93543) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:32:52.046873 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=93543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.047155 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=93543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.047436 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=93543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.047711 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=93543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.047905 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=93542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:32:52.047991 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=93543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.048198 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=93542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.048290 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=93543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.048484 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=93542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.048586 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=93543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.048793 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=93542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.049074 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=93542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.049352 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=93542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.049621 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=93542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.049890 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=93542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.066025 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=93543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.066309 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=93543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.067616 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=93542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.067921 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=93542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.070219 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=93543) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:32:52.071829 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=93542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:32:52.073446 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=93543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.073777 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=93543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.075099 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=93542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.075455 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=93542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.077980 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=93543) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:32:52.079722 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=93542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:32:52.081116 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=93543) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:32:52.083401 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=93542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:32:52.084872 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=93543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.085174 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=93543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.085456 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=93543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.085737 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=93543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.086019 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=93543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.092509 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=93543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.092625 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=93542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.092625 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=93542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.092625 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=93542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.092625 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=93542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.092625 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=93542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.092625 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=93542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.103681 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=93543) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:32:52.103973 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=93543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.104242 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=93543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.104585 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=93543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.105466 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=93542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:32:52.105761 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=93542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.106031 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=93542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.106310 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=93542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.111978 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=93543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.113660 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=93542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.113767 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=93543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.113999 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=93543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.115170 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=93542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.115443 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=93542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.117005 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=93543) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:32:52.117288 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=93543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.117583 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=93543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.117887 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=93543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.118197 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=93543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.118414 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=93542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:32:52.118703 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=93542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.118994 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=93542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.119287 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=93542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.119570 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=93542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.129502 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=93543) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:32:52.129781 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=93543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.130065 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=93543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.130345 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=93543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.130628 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=93543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.130905 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=93543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.131022 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=93542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:32:52.131182 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=93543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.131308 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=93542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.131586 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=93542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.131868 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=93542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.132152 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=93542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.132431 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=93542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.132752 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=93542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.146995 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=93543) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:32:52.148813 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=93542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:32:52.150009 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=93543) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:32:52.151872 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=93542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:32:52.152894 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=93543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.154313 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=93543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.159479 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=93542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.159569 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=93543) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:32:52.159569 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=93543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.159704 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=93542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.159777 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=93543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.159852 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=93542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:32:52.159852 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=93542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.159852 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=93542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.163310 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=93543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.164809 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=93543) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:32:52.165325 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=93542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.166779 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=93542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:32:52.167659 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=93543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.167948 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=93543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.168241 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=93543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.168525 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=93543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.169751 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=93542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.170045 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=93542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.170344 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=93542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.170635 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=93542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.176469 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=93543) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:32:52.176782 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=93543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.177064 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=93543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.178755 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=93542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:32:52.179048 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=93542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.179334 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=93542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.183896 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=93543) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:32:52.184172 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=93543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.184452 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=93543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.184754 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=93543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.185028 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=93543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.185312 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=93543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.185586 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=93543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.185861 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=93543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.186137 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=93543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.186265 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=93542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:32:52.186405 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=93543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.186550 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=93542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.186697 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=93543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.186834 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=93542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.186958 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=93543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.187111 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=93542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.187242 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=93543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.187387 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=93542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.187485 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=93543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.187667 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=93542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.187952 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=93542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.188224 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=93542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.188504 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=93542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.188806 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=93542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.189081 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=93542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.189362 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=93542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.189641 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=93542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.189904 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=93542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:32:52.221806 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55e4d921d9c0 pid: 93543 (default app) Aug 24 08:32:52.225856 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55e4d921d9c0 pid: 93542 (default app) Aug 24 08:32:52.225999 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d06ec4-e2f7-4b8f-b6cc-e8a7a208689f None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:52.226211 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d06ec4-e2f7-4b8f-b6cc-e8a7a208689f None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:52.226482 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d06ec4-e2f7-4b8f-b6cc-e8a7a208689f None None] PATH_INFO: `/v3` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:52.230676 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1/1] 10.222.0.55 () {50 vars in 791 bytes} [Wed Aug 24 08:32:52 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:32:59.900093 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba893145-7195-4514-9454-9cb244ce978d None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:59.900629 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba893145-7195-4514-9454-9cb244ce978d None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:59.900749 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba893145-7195-4514-9454-9cb244ce978d None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:59.904999 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1/2] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:32:59 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:32:59.911564 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d824e5-3aa2-4876-96d4-1a5e9e202193 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:32:59.911976 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d824e5-3aa2-4876-96d4-1a5e9e202193 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:32:59.912101 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d824e5-3aa2-4876-96d4-1a5e9e202193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:32:59.982440 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: INFO dbcounter [None req-a4d824e5-3aa2-4876-96d4-1a5e9e202193 None None] Registered counter for database keystone Aug 24 08:33:00.004425 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG oslo_db.sqlalchemy.engines [None req-a4d824e5-3aa2-4876-96d4-1a5e9e202193 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=93543) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 24 08:33:00.011229 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writer thread running {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 24 08:33:00.194564 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG passlib.handlers.bcrypt [None req-a4d824e5-3aa2-4876-96d4-1a5e9e202193 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=93543) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 24 08:33:00.194909 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG passlib.handlers.bcrypt [None req-a4d824e5-3aa2-4876-96d4-1a5e9e202193 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=93543) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 24 08:33:00.250136 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-a4d824e5-3aa2-4876-96d4-1a5e9e202193 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:33:00.251768 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-a4d824e5-3aa2-4876-96d4-1a5e9e202193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:00.382693 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 2/3] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:32:59 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 472 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 24 08:33:00.822993 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba893145-7195-4514-9454-9cb244ce978d None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:00.891894 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: INFO dbcounter [None req-ba893145-7195-4514-9454-9cb244ce978d None None] Registered counter for database keystone Aug 24 08:33:00.911554 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG oslo_db.sqlalchemy.engines [None req-ba893145-7195-4514-9454-9cb244ce978d None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=93542) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 24 08:33:00.918308 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writer thread running {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 24 08:33:01.038847 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba893145-7195-4514-9454-9cb244ce978d None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:01.043177 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08c2e4a-2ea8-4179-aaff-445c010a6fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:01.043798 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08c2e4a-2ea8-4179-aaff-445c010a6fcc admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:01.044080 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08c2e4a-2ea8-4179-aaff-445c010a6fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:01.044358 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08c2e4a-2ea8-4179-aaff-445c010a6fcc admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:01.052590 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08c2e4a-2ea8-4179-aaff-445c010a6fcc admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:01.104708 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: WARNING keystone.common.rbac_enforcer.enforcer [None req-e08c2e4a-2ea8-4179-aaff-445c010a6fcc admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 24 08:33:01.104992 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08c2e4a-2ea8-4179-aaff-445c010a6fcc admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:01.114713 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 2/4] 10.222.0.55 () {58 vars in 1324 bytes} [Wed Aug 24 08:33:00 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 08:33:01.121217 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d824e5-3aa2-4876-96d4-1a5e9e202193 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:01.136330 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d824e5-3aa2-4876-96d4-1a5e9e202193 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:01.140793 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebae63c0-063f-4692-a5cd-d6b374ea0c95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:01.141967 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebae63c0-063f-4692-a5cd-d6b374ea0c95 admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:01.142266 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebae63c0-063f-4692-a5cd-d6b374ea0c95 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:01.142543 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebae63c0-063f-4692-a5cd-d6b374ea0c95 admin admin] PATH_INFO: `/v3/projects/6ad87fa1813247aba83e93f886f34eb3` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:01.144125 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebae63c0-063f-4692-a5cd-d6b374ea0c95 admin admin] RBAC: Authorizing `identity:get_project(project_id=6ad87fa1813247aba83e93f886f34eb3)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:01.197249 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: WARNING keystone.common.rbac_enforcer.enforcer [None req-ebae63c0-063f-4692-a5cd-d6b374ea0c95 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 24 08:33:01.197602 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebae63c0-063f-4692-a5cd-d6b374ea0c95 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:01.202382 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 3/5] 10.222.0.55 () {56 vars in 1207 bytes} [Wed Aug 24 08:33:01 2022] GET /identity/v3/projects/6ad87fa1813247aba83e93f886f34eb3 => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:33:03.084307 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71e2c64-3c4c-40d2-b925-fb651e698136 None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:03.085435 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71e2c64-3c4c-40d2-b925-fb651e698136 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:03.085435 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71e2c64-3c4c-40d2-b925-fb651e698136 None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:03.089483 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 3/6] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:33:03 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:33:03.095042 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcd8272-44e8-465e-afc7-52908a1a7ff8 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:03.095414 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcd8272-44e8-465e-afc7-52908a1a7ff8 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:03.095589 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcd8272-44e8-465e-afc7-52908a1a7ff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:03.115975 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-3fcd8272-44e8-465e-afc7-52908a1a7ff8 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:33:03.117538 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-3fcd8272-44e8-465e-afc7-52908a1a7ff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:03.125431 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 4/7] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:33:03 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 24 08:33:03.389973 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71e2c64-3c4c-40d2-b925-fb651e698136 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:03.402669 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71e2c64-3c4c-40d2-b925-fb651e698136 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:03.406962 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b74169a-68d4-4f27-b8bc-e945d4058864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:03.407565 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b74169a-68d4-4f27-b8bc-e945d4058864 admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:03.407852 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b74169a-68d4-4f27-b8bc-e945d4058864 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:03.408132 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b74169a-68d4-4f27-b8bc-e945d4058864 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:03.415815 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b74169a-68d4-4f27-b8bc-e945d4058864 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:03.417635 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b74169a-68d4-4f27-b8bc-e945d4058864 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:03.426385 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 4/8] 10.222.0.55 () {58 vars in 1324 bytes} [Wed Aug 24 08:33:03 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 08:33:03.431975 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcd8272-44e8-465e-afc7-52908a1a7ff8 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:03.441088 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcd8272-44e8-465e-afc7-52908a1a7ff8 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:03.446075 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c10836-c9ff-4f85-ba52-cffb6dca0a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:03.446701 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c10836-c9ff-4f85-ba52-cffb6dca0a25 admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:03.446980 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c10836-c9ff-4f85-ba52-cffb6dca0a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:03.447252 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c10836-c9ff-4f85-ba52-cffb6dca0a25 admin admin] PATH_INFO: `/v3/users/28800687066c4de989fe1940c9a30412` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:03.448636 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c10836-c9ff-4f85-ba52-cffb6dca0a25 admin admin] RBAC: Authorizing `identity:get_user(user_id=28800687066c4de989fe1940c9a30412)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:03.450481 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c10836-c9ff-4f85-ba52-cffb6dca0a25 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:03.451976 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 5/9] 10.222.0.55 () {56 vars in 1198 bytes} [Wed Aug 24 08:33:03 2022] GET /identity/v3/users/28800687066c4de989fe1940c9a30412 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:05.422977 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a6749e-cc78-495f-bc3a-bc01a022b06a None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:05.423498 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a6749e-cc78-495f-bc3a-bc01a022b06a None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:05.423498 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a6749e-cc78-495f-bc3a-bc01a022b06a None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:05.424905 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 5/10] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:33:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:33:05.430825 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5dea23-13f2-488e-a188-d94f287e03fc None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:05.431285 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5dea23-13f2-488e-a188-d94f287e03fc None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:05.431381 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5dea23-13f2-488e-a188-d94f287e03fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:05.441887 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d8509-0a63-4b13-8189-edc0816daf1a None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:05.442177 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d8509-0a63-4b13-8189-edc0816daf1a None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:05.442453 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d8509-0a63-4b13-8189-edc0816daf1a None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:05.443741 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 6/11] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:33:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:33:05.449305 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe730e7-9a52-43e0-8ee2-7b81b87075eb None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:05.449526 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe730e7-9a52-43e0-8ee2-7b81b87075eb None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:05.449800 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe730e7-9a52-43e0-8ee2-7b81b87075eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:05.450103 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-4b5dea23-13f2-488e-a188-d94f287e03fc None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:33:05.451296 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-4b5dea23-13f2-488e-a188-d94f287e03fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:05.490299 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG passlib.handlers.bcrypt [None req-bbe730e7-9a52-43e0-8ee2-7b81b87075eb None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=93542) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 24 08:33:05.490715 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG passlib.handlers.bcrypt [None req-bbe730e7-9a52-43e0-8ee2-7b81b87075eb None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=93542) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 24 08:33:05.530341 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 6/12] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:33:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:33:05.537099 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-bbe730e7-9a52-43e0-8ee2-7b81b87075eb None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:33:05.538716 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-bbe730e7-9a52-43e0-8ee2-7b81b87075eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:05.637737 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 7/13] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:33:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 08:33:05.797231 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5dea23-13f2-488e-a188-d94f287e03fc None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:05.820926 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:05.901402 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5dea23-13f2-488e-a188-d94f287e03fc None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:05.908035 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe730e7-9a52-43e0-8ee2-7b81b87075eb None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:05.936491 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcb03c1-f5ef-4d7e-ae71-d82a47be5ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:05.937177 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcb03c1-f5ef-4d7e-ae71-d82a47be5ba3 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:05.937456 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcb03c1-f5ef-4d7e-ae71-d82a47be5ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:05.937728 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcb03c1-f5ef-4d7e-ae71-d82a47be5ba3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:05.949181 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcb03c1-f5ef-4d7e-ae71-d82a47be5ba3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:33:05.957562 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-4fcb03c1-f5ef-4d7e-ae71-d82a47be5ba3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:33:05.957562 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:33:05.957562 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:33:05.957562 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:33:05.957562 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:33:05.957562 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:33:05.957562 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:33:05.957562 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:33:05.957562 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:33:05.957562 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:33:05.957562 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:33:05.957562 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:33:05.957562 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:33:05.957562 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:33:05.957562 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:33:05.957562 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:33:05.958631 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:33:05.958631 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:33:05.958631 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:33:05.958631 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:33:05.958631 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:33:05.958631 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:33:05.958631 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:33:05.958631 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:33:05.958631 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:33:05.958631 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:33:05.958631 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:33:05.958631 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:33:05.958631 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:33:05.958631 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:33:05.958631 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:33:05.958631 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:33:05.961611 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:33:05.961611 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:33:05.961611 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:33:05.961611 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:33:05.961611 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:33:05.961611 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:33:05.961611 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:33:05.961611 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:33:05.961611 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:33:05.961611 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:33:05.961611 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:33:05.961611 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:33:05.961611 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 7/14] 10.222.0.55 () {56 vars in 1096 bytes} [Wed Aug 24 08:33:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:33:05.965281 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcb03c1-f5ef-4d7e-ae71-d82a47be5ba3 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:06.019613 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe730e7-9a52-43e0-8ee2-7b81b87075eb None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:06.022018 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:06.058493 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:06.067966 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc169018-bcf8-43b2-95cc-cedc64bd98e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:06.069347 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc169018-bcf8-43b2-95cc-cedc64bd98e1 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:06.069799 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc169018-bcf8-43b2-95cc-cedc64bd98e1 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:06.070247 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc169018-bcf8-43b2-95cc-cedc64bd98e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:06.086717 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcb03c1-f5ef-4d7e-ae71-d82a47be5ba3 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:06.137958 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ddc393-bd75-4a82-b5ec-2cff5993cf2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:06.138721 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ddc393-bd75-4a82-b5ec-2cff5993cf2f None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:06.139105 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ddc393-bd75-4a82-b5ec-2cff5993cf2f None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:06.139483 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ddc393-bd75-4a82-b5ec-2cff5993cf2f None admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:06.140295 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ddc393-bd75-4a82-b5ec-2cff5993cf2f None admin] RBAC: Adding query filter params (name=admin) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:33:06.140785 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ddc393-bd75-4a82-b5ec-2cff5993cf2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:06.142305 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ddc393-bd75-4a82-b5ec-2cff5993cf2f None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:06.154892 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 8/15] 10.222.0.55 () {56 vars in 1099 bytes} [Wed Aug 24 08:33:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:33:06.159615 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc169018-bcf8-43b2-95cc-cedc64bd98e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:06.160357 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ddc393-bd75-4a82-b5ec-2cff5993cf2f None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:06.161166 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc169018-bcf8-43b2-95cc-cedc64bd98e1 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:06.258555 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ddc393-bd75-4a82-b5ec-2cff5993cf2f None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:06.259020 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 8/16] 10.222.0.55 () {58 vars in 1282 bytes} [Wed Aug 24 08:33:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 08:33:06.264768 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc169018-bcf8-43b2-95cc-cedc64bd98e1 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:06.267515 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:06.293846 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757c714e-3d02-45f8-aab3-230c06bcbfe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:06.294476 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757c714e-3d02-45f8-aab3-230c06bcbfe2 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:06.294752 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757c714e-3d02-45f8-aab3-230c06bcbfe2 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:06.295020 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757c714e-3d02-45f8-aab3-230c06bcbfe2 None admin] PATH_INFO: `/v3/users/28800687066c4de989fe1940c9a30412` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:06.296155 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757c714e-3d02-45f8-aab3-230c06bcbfe2 None admin] RBAC: Authorizing `identity:get_user(user_id=28800687066c4de989fe1940c9a30412)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:06.297628 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757c714e-3d02-45f8-aab3-230c06bcbfe2 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:06.299184 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 9/17] 10.222.0.55 () {56 vars in 1177 bytes} [Wed Aug 24 08:33:06 2022] GET /identity/v3/users/28800687066c4de989fe1940c9a30412 => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:33:06.304685 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757c714e-3d02-45f8-aab3-230c06bcbfe2 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:06.318875 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:06.364443 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc169018-bcf8-43b2-95cc-cedc64bd98e1 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:06.400368 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa8dd90-a041-4e0a-8a3f-287b61693683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:06.401292 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa8dd90-a041-4e0a-8a3f-287b61693683 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:06.401572 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa8dd90-a041-4e0a-8a3f-287b61693683 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:06.401847 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa8dd90-a041-4e0a-8a3f-287b61693683 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:06.409988 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757c714e-3d02-45f8-aab3-230c06bcbfe2 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:06.420518 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa8dd90-a041-4e0a-8a3f-287b61693683 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:06.422007 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa8dd90-a041-4e0a-8a3f-287b61693683 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:06.440160 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-6fa8dd90-a041-4e0a-8a3f-287b61693683 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:33:06.440160 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:33:06.440160 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:33:06.440160 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:33:06.440160 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:33:06.440160 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:33:06.440160 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:33:06.440160 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:33:06.440160 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:33:06.440160 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:33:06.440160 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:33:06.440160 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:33:06.440160 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:33:06.440160 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:33:06.440160 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:33:06.440160 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:33:06.441892 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:33:06.441892 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:33:06.441892 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:33:06.441892 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:33:06.441892 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:33:06.441892 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:33:06.441892 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:33:06.441892 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:33:06.441892 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:33:06.441892 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:33:06.441892 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:33:06.441892 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:33:06.441892 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:33:06.441892 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:33:06.441892 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:33:06.441892 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:33:06.441892 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:33:06.442987 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:33:06.442987 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:33:06.442987 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:33:06.442987 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 9/18] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:33:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:33:06.445507 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48667298-450f-4616-b836-7e36369211de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:06.446148 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48667298-450f-4616-b836-7e36369211de None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:06.446429 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48667298-450f-4616-b836-7e36369211de None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:06.446793 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48667298-450f-4616-b836-7e36369211de None admin] PATH_INFO: `/v3/domains/default` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:06.447561 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa8dd90-a041-4e0a-8a3f-287b61693683 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:06.447926 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48667298-450f-4616-b836-7e36369211de None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:06.449404 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48667298-450f-4616-b836-7e36369211de None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:06.451033 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 10/19] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:33:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:33:06.456678 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48667298-450f-4616-b836-7e36369211de None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:06.504355 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:06.537618 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:06.550002 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa8dd90-a041-4e0a-8a3f-287b61693683 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:06.557070 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48667298-450f-4616-b836-7e36369211de None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:06.584427 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7bbb87-1f3f-4a65-ab60-b4c1d3b45d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:06.585611 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7bbb87-1f3f-4a65-ab60-b4c1d3b45d35 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:06.585893 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7bbb87-1f3f-4a65-ab60-b4c1d3b45d35 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:06.586174 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7bbb87-1f3f-4a65-ab60-b4c1d3b45d35 None admin] PATH_INFO: `/v3/domains` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:06.586818 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7bbb87-1f3f-4a65-ab60-b4c1d3b45d35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:33:06.587122 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7bbb87-1f3f-4a65-ab60-b4c1d3b45d35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:06.588511 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7bbb87-1f3f-4a65-ab60-b4c1d3b45d35 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:06.593594 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7b4631-49c0-4894-a19a-17eb16d98b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:06.594191 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7b4631-49c0-4894-a19a-17eb16d98b70 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:06.594466 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7b4631-49c0-4894-a19a-17eb16d98b70 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:06.594736 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7b4631-49c0-4894-a19a-17eb16d98b70 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:06.595451 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7b4631-49c0-4894-a19a-17eb16d98b70 None admin] RBAC: Adding query filter params (role.id=b870cd1acaa744e1ae1d58b9ac283821, scope.domain.id=default, user.id=28800687066c4de989fe1940c9a30412) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:33:06.595754 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7b4631-49c0-4894-a19a-17eb16d98b70 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:06.597224 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7b4631-49c0-4894-a19a-17eb16d98b70 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:06.604305 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 11/20] 10.222.0.55 () {56 vars in 1322 bytes} [Wed Aug 24 08:33:06 2022] GET /identity/v3/role_assignments?user.id=28800687066c4de989fe1940c9a30412&scope.domain.id=default&role.id=b870cd1acaa744e1ae1d58b9ac283821 => generated 233 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:33:06.608998 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 10/21] 10.222.0.55 () {56 vars in 1109 bytes} [Wed Aug 24 08:33:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:33:08.755042 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082d51d7-f2fb-4bea-a0f3-57ffa4226876 None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:08.756084 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082d51d7-f2fb-4bea-a0f3-57ffa4226876 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:08.756590 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082d51d7-f2fb-4bea-a0f3-57ffa4226876 None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:08.758648 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 12/22] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:33:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:33:08.764479 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a4328f-36d7-43c6-af66-18f29a4fc884 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:08.765345 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a4328f-36d7-43c6-af66-18f29a4fc884 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:08.765891 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a4328f-36d7-43c6-af66-18f29a4fc884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:08.793506 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-e6a4328f-36d7-43c6-af66-18f29a4fc884 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:33:08.795045 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-e6a4328f-36d7-43c6-af66-18f29a4fc884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:08.883424 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 11/23] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:33:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:33:08.898923 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a05454-f541-42fc-aee1-18ddc383cda8 None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:08.899530 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a05454-f541-42fc-aee1-18ddc383cda8 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:08.900016 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a05454-f541-42fc-aee1-18ddc383cda8 None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:08.901569 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 13/24] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:33:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:33:08.906898 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48b1d5e-735c-4f1b-bacf-618d14c371fb None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:08.907395 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48b1d5e-735c-4f1b-bacf-618d14c371fb None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:08.907860 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48b1d5e-735c-4f1b-bacf-618d14c371fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:08.941656 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-c48b1d5e-735c-4f1b-bacf-618d14c371fb None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:33:08.942878 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-c48b1d5e-735c-4f1b-bacf-618d14c371fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:08.997136 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:09.034910 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 12/25] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:33:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:33:09.192918 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95fd0c0-f0c4-4021-8181-f3c335fbc693 None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:09.198745 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95fd0c0-f0c4-4021-8181-f3c335fbc693 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:09.199278 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95fd0c0-f0c4-4021-8181-f3c335fbc693 None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:09.206190 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 14/26] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:33:09 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:33:09.212126 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18586f0-7ce7-410e-9355-cc5a203da5f5 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:09.212519 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18586f0-7ce7-410e-9355-cc5a203da5f5 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:09.212823 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18586f0-7ce7-410e-9355-cc5a203da5f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:09.242108 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-e18586f0-7ce7-410e-9355-cc5a203da5f5 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:33:09.243308 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-e18586f0-7ce7-410e-9355-cc5a203da5f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:09.321247 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95fd0c0-f0c4-4021-8181-f3c335fbc693 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:09.326939 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 13/27] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:33:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:33:09.531830 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95fd0c0-f0c4-4021-8181-f3c335fbc693 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:09.582892 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18586f0-7ce7-410e-9355-cc5a203da5f5 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:09.595383 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beccfa43-9dfa-48c6-880b-f1289f2ad51f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:09.596035 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beccfa43-9dfa-48c6-880b-f1289f2ad51f None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:09.596315 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beccfa43-9dfa-48c6-880b-f1289f2ad51f None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:09.604648 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beccfa43-9dfa-48c6-880b-f1289f2ad51f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:09.630867 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beccfa43-9dfa-48c6-880b-f1289f2ad51f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:09.632890 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beccfa43-9dfa-48c6-880b-f1289f2ad51f None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:09.644878 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:09.663721 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-beccfa43-9dfa-48c6-880b-f1289f2ad51f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:33:09.663721 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:33:09.663721 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:33:09.663721 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:33:09.663721 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:33:09.663721 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:33:09.663721 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:33:09.663721 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:33:09.663721 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:33:09.663721 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:33:09.663721 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:33:09.663721 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:33:09.663721 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:33:09.663721 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:33:09.663721 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:33:09.663721 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:33:09.664659 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:33:09.664659 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:33:09.664659 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:33:09.664659 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:33:09.664659 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:33:09.664659 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:33:09.664659 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:33:09.664659 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:33:09.664659 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:33:09.664659 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:33:09.664659 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:33:09.664659 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:33:09.664659 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:33:09.664659 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:33:09.664659 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:33:09.664659 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:33:09.664659 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:33:09.665465 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:33:09.665465 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:33:09.665465 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:33:09.665465 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 15/28] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:33:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:33:09.666078 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d54cc3-2a64-474d-9816-66095c6d4562 None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:09.667153 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d54cc3-2a64-474d-9816-66095c6d4562 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:09.667153 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d54cc3-2a64-474d-9816-66095c6d4562 None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:09.667852 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 16/29] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:33:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:33:09.678200 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56df0f13-976e-4de1-840f-87c258e87751 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:09.678490 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56df0f13-976e-4de1-840f-87c258e87751 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:09.678772 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56df0f13-976e-4de1-840f-87c258e87751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:09.706046 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-56df0f13-976e-4de1-840f-87c258e87751 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:33:09.707546 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-56df0f13-976e-4de1-840f-87c258e87751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:09.749259 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18586f0-7ce7-410e-9355-cc5a203da5f5 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:09.768937 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:09.788300 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 17/30] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:33:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:33:09.789954 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56df0f13-976e-4de1-840f-87c258e87751 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:09.803211 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f827d9-47b8-4cdd-a11a-d592ede8a3ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:09.805033 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f827d9-47b8-4cdd-a11a-d592ede8a3ad None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:09.805934 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f827d9-47b8-4cdd-a11a-d592ede8a3ad None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:09.807354 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f827d9-47b8-4cdd-a11a-d592ede8a3ad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:09.831177 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f827d9-47b8-4cdd-a11a-d592ede8a3ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:33:09.840461 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-63f827d9-47b8-4cdd-a11a-d592ede8a3ad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:33:09.840461 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:33:09.840461 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:33:09.840461 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:33:09.840461 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:33:09.840461 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:33:09.840461 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:33:09.840461 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:33:09.840461 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:33:09.840461 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:33:09.840461 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:33:09.840461 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:33:09.840461 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:33:09.840461 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:33:09.840461 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:33:09.840461 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:33:09.841314 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:33:09.841314 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:33:09.841314 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:33:09.841314 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:33:09.841314 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:33:09.841314 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:33:09.841314 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:33:09.841314 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:33:09.841314 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:33:09.841314 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:33:09.841314 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:33:09.841314 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:33:09.841314 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:33:09.841314 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:33:09.841314 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:33:09.841314 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:33:09.842081 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:33:09.842081 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:33:09.842081 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:33:09.842081 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:33:09.842081 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:33:09.842081 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:33:09.842081 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:33:09.842081 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:33:09.842081 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:33:09.842081 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:33:09.842081 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:33:09.842081 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:33:09.844574 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 14/31] 10.222.0.55 () {56 vars in 1096 bytes} [Wed Aug 24 08:33:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:33:09.845551 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f827d9-47b8-4cdd-a11a-d592ede8a3ad None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:09.967498 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56df0f13-976e-4de1-840f-87c258e87751 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:09.999518 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:10.010382 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f827d9-47b8-4cdd-a11a-d592ede8a3ad None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:10.022209 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15bcd74-624b-4b02-94c0-f3b1186e18e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:10.022881 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15bcd74-624b-4b02-94c0-f3b1186e18e9 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:10.023182 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15bcd74-624b-4b02-94c0-f3b1186e18e9 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:10.023464 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15bcd74-624b-4b02-94c0-f3b1186e18e9 None admin] PATH_INFO: `/v3/domains` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:10.024136 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15bcd74-624b-4b02-94c0-f3b1186e18e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:33:10.024442 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15bcd74-624b-4b02-94c0-f3b1186e18e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:10.025873 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15bcd74-624b-4b02-94c0-f3b1186e18e9 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:10.054350 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a83752-b319-4eaa-b0d6-dbb67d943287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:10.055478 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a83752-b319-4eaa-b0d6-dbb67d943287 None admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:10.057628 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a83752-b319-4eaa-b0d6-dbb67d943287 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:10.058098 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a83752-b319-4eaa-b0d6-dbb67d943287 None admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:10.059295 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 18/32] 10.222.0.55 () {56 vars in 1109 bytes} [Wed Aug 24 08:33:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:10.060680 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a83752-b319-4eaa-b0d6-dbb67d943287 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:10.061962 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15bcd74-624b-4b02-94c0-f3b1186e18e9 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:10.062966 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a83752-b319-4eaa-b0d6-dbb67d943287 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:10.186783 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15bcd74-624b-4b02-94c0-f3b1186e18e9 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:10.224961 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e7e8ed-05ea-4354-bbca-5252b1e2a591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:10.225566 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e7e8ed-05ea-4354-bbca-5252b1e2a591 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:10.226343 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e7e8ed-05ea-4354-bbca-5252b1e2a591 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:10.226623 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e7e8ed-05ea-4354-bbca-5252b1e2a591 None admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:10.227300 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e7e8ed-05ea-4354-bbca-5252b1e2a591 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:33:10.227613 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e7e8ed-05ea-4354-bbca-5252b1e2a591 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:10.229081 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e7e8ed-05ea-4354-bbca-5252b1e2a591 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:10.237839 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 19/33] 10.222.0.55 () {56 vars in 1099 bytes} [Wed Aug 24 08:33:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:10.243759 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e7e8ed-05ea-4354-bbca-5252b1e2a591 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:10.296466 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:10.331300 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 15/34] 10.222.0.55 () {60 vars in 1131 bytes} [Wed Aug 24 08:33:09 2022] POST /identity/v3/roles => generated 219 bytes in 486 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:33:10.332725 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a83752-b319-4eaa-b0d6-dbb67d943287 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:10.353713 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e7e8ed-05ea-4354-bbca-5252b1e2a591 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:10.386427 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:10.395446 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d9c486-685d-480f-bccf-16e045e37505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:10.396077 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d9c486-685d-480f-bccf-16e045e37505 None admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:10.396369 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d9c486-685d-480f-bccf-16e045e37505 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:10.396678 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d9c486-685d-480f-bccf-16e045e37505 None admin] PATH_INFO: `/v3/projects` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:10.397350 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d9c486-685d-480f-bccf-16e045e37505 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:10.398867 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d9c486-685d-480f-bccf-16e045e37505 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:10.456236 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a83752-b319-4eaa-b0d6-dbb67d943287 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:10.494737 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4c5d11-6c64-4b2f-9d26-07f6517583c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:10.497963 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4c5d11-6c64-4b2f-9d26-07f6517583c0 None admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:10.497963 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4c5d11-6c64-4b2f-9d26-07f6517583c0 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:10.497963 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4c5d11-6c64-4b2f-9d26-07f6517583c0 None admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:10.497963 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4c5d11-6c64-4b2f-9d26-07f6517583c0 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:10.498296 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4c5d11-6c64-4b2f-9d26-07f6517583c0 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:10.604084 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 16/35] 10.222.0.55 () {60 vars in 1131 bytes} [Wed Aug 24 08:33:10 2022] POST /identity/v3/roles => generated 225 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:33:10.605037 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4c5d11-6c64-4b2f-9d26-07f6517583c0 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:10.651690 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 20/36] 10.222.0.55 () {60 vars in 1141 bytes} [Wed Aug 24 08:33:10 2022] POST /identity/v3/projects => generated 301 bytes in 413 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:33:10.653401 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4ec8f4-522b-4a45-a208-622d21dff641 None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:10.653702 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4ec8f4-522b-4a45-a208-622d21dff641 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:10.653982 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4ec8f4-522b-4a45-a208-622d21dff641 None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:10.655467 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 21/37] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:33:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:33:10.656626 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309e5391-8d4c-47cf-97ee-77efdef6672c None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:10.656919 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309e5391-8d4c-47cf-97ee-77efdef6672c None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:10.657198 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309e5391-8d4c-47cf-97ee-77efdef6672c None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:10.663202 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 22/38] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:33:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:33:10.663202 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fdf9a4-06a0-40df-9afb-aff01973ed45 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:10.663202 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fdf9a4-06a0-40df-9afb-aff01973ed45 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:10.663202 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fdf9a4-06a0-40df-9afb-aff01973ed45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:10.685979 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-64fdf9a4-06a0-40df-9afb-aff01973ed45 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:33:10.688173 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-64fdf9a4-06a0-40df-9afb-aff01973ed45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:10.720639 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:10.741668 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4c5d11-6c64-4b2f-9d26-07f6517583c0 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:10.789890 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 23/39] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:33:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 08:33:10.792516 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65644329-d153-4e83-9a5b-b858a1d54dc0 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:10.792833 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65644329-d153-4e83-9a5b-b858a1d54dc0 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:10.793114 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65644329-d153-4e83-9a5b-b858a1d54dc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:10.801803 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbe7318-ea3d-4466-9379-aa74abb3ddce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:10.802460 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbe7318-ea3d-4466-9379-aa74abb3ddce None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:10.802741 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbe7318-ea3d-4466-9379-aa74abb3ddce None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:10.803017 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbe7318-ea3d-4466-9379-aa74abb3ddce None admin] PATH_INFO: `/v3/users/28800687066c4de989fe1940c9a30412` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:10.805814 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbe7318-ea3d-4466-9379-aa74abb3ddce None admin] RBAC: Authorizing `identity:get_user(user_id=28800687066c4de989fe1940c9a30412)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:10.807663 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbe7318-ea3d-4466-9379-aa74abb3ddce None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:10.810765 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 17/40] 10.222.0.55 () {56 vars in 1177 bytes} [Wed Aug 24 08:33:10 2022] GET /identity/v3/users/28800687066c4de989fe1940c9a30412 => generated 247 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:10.821842 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbe7318-ea3d-4466-9379-aa74abb3ddce None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:10.822041 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-65644329-d153-4e83-9a5b-b858a1d54dc0 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:33:10.823362 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-65644329-d153-4e83-9a5b-b858a1d54dc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:10.856234 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:10.924213 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 24/41] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:33:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:33:10.955502 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbe7318-ea3d-4466-9379-aa74abb3ddce None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:10.990318 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd5577b-97c9-4940-a2de-5ce0f0159691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:10.990933 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd5577b-97c9-4940-a2de-5ce0f0159691 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:10.991229 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd5577b-97c9-4940-a2de-5ce0f0159691 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:10.991518 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd5577b-97c9-4940-a2de-5ce0f0159691 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:10.992759 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd5577b-97c9-4940-a2de-5ce0f0159691 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:10.994458 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd5577b-97c9-4940-a2de-5ce0f0159691 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:10.996210 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 18/42] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:33:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:11.001680 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65644329-d153-4e83-9a5b-b858a1d54dc0 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:11.080088 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd5577b-97c9-4940-a2de-5ce0f0159691 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:11.106864 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:11.107918 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65644329-d153-4e83-9a5b-b858a1d54dc0 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:11.146670 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff48e73-8473-4052-86dc-b548dc02bef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:11.147330 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff48e73-8473-4052-86dc-b548dc02bef9 None admin] REQUEST_METHOD: `PUT` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:11.147612 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff48e73-8473-4052-86dc-b548dc02bef9 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:11.148432 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff48e73-8473-4052-86dc-b548dc02bef9 None admin] PATH_INFO: `/v3/domains/default/users/28800687066c4de989fe1940c9a30412/roles/b870cd1acaa744e1ae1d58b9ac283821` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:11.150493 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff48e73-8473-4052-86dc-b548dc02bef9 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=28800687066c4de989fe1940c9a30412, role_id=b870cd1acaa744e1ae1d58b9ac283821)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:11.152086 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff48e73-8473-4052-86dc-b548dc02bef9 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:11.186101 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd5577b-97c9-4940-a2de-5ce0f0159691 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:11.221881 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f457062-3903-4deb-ac6c-b66b316ffe61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:11.222502 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f457062-3903-4deb-ac6c-b66b316ffe61 None admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:11.222785 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f457062-3903-4deb-ac6c-b66b316ffe61 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:11.223059 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f457062-3903-4deb-ac6c-b66b316ffe61 None admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:11.223734 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f457062-3903-4deb-ac6c-b66b316ffe61 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:11.225194 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f457062-3903-4deb-ac6c-b66b316ffe61 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:11.259613 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 25/43] 10.222.0.55 () {58 vars in 1361 bytes} [Wed Aug 24 08:33:11 2022] PUT /identity/v3/domains/default/users/28800687066c4de989fe1940c9a30412/roles/b870cd1acaa744e1ae1d58b9ac283821 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:33:11.259613 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff48e73-8473-4052-86dc-b548dc02bef9 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:11.313202 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 19/44] 10.222.0.55 () {60 vars in 1131 bytes} [Wed Aug 24 08:33:11 2022] POST /identity/v3/roles => generated 223 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:33:11.393852 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:11.483260 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff48e73-8473-4052-86dc-b548dc02bef9 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:11.520657 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d095e29-b3cd-499e-94b4-46e5f96ab1d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:11.521177 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d095e29-b3cd-499e-94b4-46e5f96ab1d9 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:11.521461 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d095e29-b3cd-499e-94b4-46e5f96ab1d9 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:11.521737 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d095e29-b3cd-499e-94b4-46e5f96ab1d9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:11.523832 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d095e29-b3cd-499e-94b4-46e5f96ab1d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:11.524367 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d095e29-b3cd-499e-94b4-46e5f96ab1d9 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:11.526286 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 26/45] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:33:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:11.531868 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f457062-3903-4deb-ac6c-b66b316ffe61 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:11.616181 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:11.634381 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f457062-3903-4deb-ac6c-b66b316ffe61 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:11.668117 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52effa8f-efa6-4d38-986b-165326ed8874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:11.668766 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52effa8f-efa6-4d38-986b-165326ed8874 None admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:11.669050 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52effa8f-efa6-4d38-986b-165326ed8874 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:11.669327 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52effa8f-efa6-4d38-986b-165326ed8874 None admin] PATH_INFO: `/v3/projects` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:11.669982 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52effa8f-efa6-4d38-986b-165326ed8874 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:11.671463 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52effa8f-efa6-4d38-986b-165326ed8874 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:11.780658 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 20/46] 10.222.0.55 () {60 vars in 1141 bytes} [Wed Aug 24 08:33:11 2022] POST /identity/v3/projects => generated 312 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:33:13.177229 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df65a555-cdaa-4beb-8508-607b20b52966 None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:13.218681 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df65a555-cdaa-4beb-8508-607b20b52966 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:13.218681 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df65a555-cdaa-4beb-8508-607b20b52966 None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:13.218681 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 27/47] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:33:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:33:13.219516 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd0c6e0-58a3-4cbf-903c-35f7af2cd8d3 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:13.219516 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd0c6e0-58a3-4cbf-903c-35f7af2cd8d3 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:13.219516 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd0c6e0-58a3-4cbf-903c-35f7af2cd8d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:13.219516 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-ccd0c6e0-58a3-4cbf-903c-35f7af2cd8d3 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:33:13.219516 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-ccd0c6e0-58a3-4cbf-903c-35f7af2cd8d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:13.275441 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 21/48] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:33:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:33:13.542423 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df65a555-cdaa-4beb-8508-607b20b52966 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:13.632605 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:13.645151 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df65a555-cdaa-4beb-8508-607b20b52966 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:13.658177 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e97b53a-f627-4b93-b4ec-b6c72936c2d0 None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:13.658465 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e97b53a-f627-4b93-b4ec-b6c72936c2d0 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:13.658746 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e97b53a-f627-4b93-b4ec-b6c72936c2d0 None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:13.660109 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 22/49] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:33:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:33:13.666041 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e16790e-ed61-4ca2-9982-8f1cc7d8c68e None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:13.666326 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e16790e-ed61-4ca2-9982-8f1cc7d8c68e None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:13.666606 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e16790e-ed61-4ca2-9982-8f1cc7d8c68e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:13.684179 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-1e16790e-ed61-4ca2-9982-8f1cc7d8c68e None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:33:13.685096 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aca506-01d9-4899-9c4b-aeaf2a5dcbc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:13.685433 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-1e16790e-ed61-4ca2-9982-8f1cc7d8c68e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:13.685745 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aca506-01d9-4899-9c4b-aeaf2a5dcbc8 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:13.686029 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aca506-01d9-4899-9c4b-aeaf2a5dcbc8 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:13.686308 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aca506-01d9-4899-9c4b-aeaf2a5dcbc8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:13.697191 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52aca506-01d9-4899-9c4b-aeaf2a5dcbc8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:33:13.701893 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-52aca506-01d9-4899-9c4b-aeaf2a5dcbc8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:33:13.701893 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:33:13.701893 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:33:13.701893 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:33:13.701893 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:33:13.701893 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:33:13.701893 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:33:13.701893 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:33:13.701893 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:33:13.701893 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:33:13.701893 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:33:13.701893 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:33:13.701893 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:33:13.701893 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:33:13.701893 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:33:13.701893 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:33:13.702821 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:33:13.702821 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:33:13.702821 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:33:13.702821 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:33:13.702821 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:33:13.702821 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:33:13.702821 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:33:13.702821 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:33:13.702821 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:33:13.702821 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:33:13.702821 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:33:13.702821 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:33:13.702821 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:33:13.702821 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:33:13.702821 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:33:13.702821 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:33:13.705330 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:33:13.705330 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:33:13.705330 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:33:13.705330 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:33:13.705330 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:33:13.705330 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:33:13.705330 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:33:13.705330 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:33:13.705330 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:33:13.705330 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:33:13.705330 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:33:13.705330 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:33:13.705330 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 28/50] 10.222.0.55 () {56 vars in 1096 bytes} [Wed Aug 24 08:33:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:33:13.708819 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aca506-01d9-4899-9c4b-aeaf2a5dcbc8 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:13.743733 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:13.751728 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 23/51] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:33:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 08:33:13.808068 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aca506-01d9-4899-9c4b-aeaf2a5dcbc8 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:13.842911 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80f6433-5d92-43a6-97b6-c56648539a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:13.843514 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80f6433-5d92-43a6-97b6-c56648539a47 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:13.843794 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80f6433-5d92-43a6-97b6-c56648539a47 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:13.844165 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80f6433-5d92-43a6-97b6-c56648539a47 None admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:13.844756 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80f6433-5d92-43a6-97b6-c56648539a47 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:33:13.845058 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80f6433-5d92-43a6-97b6-c56648539a47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:13.846440 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80f6433-5d92-43a6-97b6-c56648539a47 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:13.854502 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 29/52] 10.222.0.55 () {56 vars in 1099 bytes} [Wed Aug 24 08:33:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:33:13.860152 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e16790e-ed61-4ca2-9982-8f1cc7d8c68e None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:13.957063 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e16790e-ed61-4ca2-9982-8f1cc7d8c68e None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:13.991079 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4d6702-19da-460c-ac90-db24d0d098ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:13.991712 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4d6702-19da-460c-ac90-db24d0d098ec None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:13.991993 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4d6702-19da-460c-ac90-db24d0d098ec None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:13.992268 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4d6702-19da-460c-ac90-db24d0d098ec None admin] PATH_INFO: `/v3/users/28800687066c4de989fe1940c9a30412` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:13.993412 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4d6702-19da-460c-ac90-db24d0d098ec None admin] RBAC: Authorizing `identity:get_user(user_id=28800687066c4de989fe1940c9a30412)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:13.995117 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4d6702-19da-460c-ac90-db24d0d098ec None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:13.996718 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 24/53] 10.222.0.55 () {56 vars in 1177 bytes} [Wed Aug 24 08:33:13 2022] GET /identity/v3/users/28800687066c4de989fe1940c9a30412 => generated 247 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:14.002739 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80f6433-5d92-43a6-97b6-c56648539a47 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:14.015736 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4d6702-19da-460c-ac90-db24d0d098ec None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:14.016222 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:14.107783 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80f6433-5d92-43a6-97b6-c56648539a47 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:14.144089 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c5b28e-75b6-4f01-bf33-56a91e4764fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:14.144773 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c5b28e-75b6-4f01-bf33-56a91e4764fe None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:14.145053 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c5b28e-75b6-4f01-bf33-56a91e4764fe None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:14.145331 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c5b28e-75b6-4f01-bf33-56a91e4764fe None admin] PATH_INFO: `/v3/domains/default` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:14.146498 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c5b28e-75b6-4f01-bf33-56a91e4764fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:14.148023 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c5b28e-75b6-4f01-bf33-56a91e4764fe None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:14.150127 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 30/54] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:33:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:14.156153 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c5b28e-75b6-4f01-bf33-56a91e4764fe None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:14.187748 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:14.226610 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4d6702-19da-460c-ac90-db24d0d098ec None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:14.236171 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:14.258113 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c5b28e-75b6-4f01-bf33-56a91e4764fe None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:14.262206 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673e35b7-dbc2-4a9f-8948-092e3432d3b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:14.262873 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673e35b7-dbc2-4a9f-8948-092e3432d3b7 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:14.263159 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673e35b7-dbc2-4a9f-8948-092e3432d3b7 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:14.263436 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673e35b7-dbc2-4a9f-8948-092e3432d3b7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:14.264634 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673e35b7-dbc2-4a9f-8948-092e3432d3b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:14.266089 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673e35b7-dbc2-4a9f-8948-092e3432d3b7 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:14.267652 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 25/55] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:33:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:33:14.273122 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673e35b7-dbc2-4a9f-8948-092e3432d3b7 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:14.291885 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268ccf08-dae6-436a-aefc-72e456c239cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:14.292496 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268ccf08-dae6-436a-aefc-72e456c239cf None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:14.292803 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268ccf08-dae6-436a-aefc-72e456c239cf None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:14.293079 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268ccf08-dae6-436a-aefc-72e456c239cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:14.293808 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268ccf08-dae6-436a-aefc-72e456c239cf None admin] RBAC: Adding query filter params (role.id=b870cd1acaa744e1ae1d58b9ac283821, scope.domain.id=default, user.id=28800687066c4de989fe1940c9a30412) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:33:14.294114 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268ccf08-dae6-436a-aefc-72e456c239cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:14.295560 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268ccf08-dae6-436a-aefc-72e456c239cf None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:14.301321 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 31/56] 10.222.0.55 () {56 vars in 1322 bytes} [Wed Aug 24 08:33:14 2022] GET /identity/v3/role_assignments?user.id=28800687066c4de989fe1940c9a30412&scope.domain.id=default&role.id=b870cd1acaa744e1ae1d58b9ac283821 => generated 530 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:33:14.377981 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673e35b7-dbc2-4a9f-8948-092e3432d3b7 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:14.408449 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:14.414689 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e6aa2d-8a12-4ada-8680-58f5f69666ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:14.415339 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e6aa2d-8a12-4ada-8680-58f5f69666ce None admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:14.415621 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e6aa2d-8a12-4ada-8680-58f5f69666ce None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:14.415908 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e6aa2d-8a12-4ada-8680-58f5f69666ce None admin] PATH_INFO: `/v3/projects` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:14.416611 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e6aa2d-8a12-4ada-8680-58f5f69666ce None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:14.422384 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e6aa2d-8a12-4ada-8680-58f5f69666ce None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:14.507079 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 26/57] 10.222.0.55 () {60 vars in 1140 bytes} [Wed Aug 24 08:33:14 2022] POST /identity/v3/projects => generated 298 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:33:16.394698 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8abab2e-90b7-4d53-8ca5-a359e6698557 None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:16.395234 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8abab2e-90b7-4d53-8ca5-a359e6698557 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:16.395234 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8abab2e-90b7-4d53-8ca5-a359e6698557 None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:16.396624 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 32/58] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:33:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:33:16.402118 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa26e0e-216a-46f3-af9b-6a4ca8371266 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:16.403635 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa26e0e-216a-46f3-af9b-6a4ca8371266 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:16.403635 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa26e0e-216a-46f3-af9b-6a4ca8371266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:16.420719 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-9aa26e0e-216a-46f3-af9b-6a4ca8371266 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:33:16.421932 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-9aa26e0e-216a-46f3-af9b-6a4ca8371266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:16.487941 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 27/59] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:33:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:33:16.764038 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8abab2e-90b7-4d53-8ca5-a359e6698557 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:16.869723 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8abab2e-90b7-4d53-8ca5-a359e6698557 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:16.903678 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca585350-5778-4482-ad80-6adffb0f82f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:16.904299 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca585350-5778-4482-ad80-6adffb0f82f2 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:16.904601 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca585350-5778-4482-ad80-6adffb0f82f2 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:16.904876 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca585350-5778-4482-ad80-6adffb0f82f2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:16.905984 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca585350-5778-4482-ad80-6adffb0f82f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:16.907414 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca585350-5778-4482-ad80-6adffb0f82f2 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:16.908998 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 33/60] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:33:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:16.914546 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa26e0e-216a-46f3-af9b-6a4ca8371266 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:17.010452 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa26e0e-216a-46f3-af9b-6a4ca8371266 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:17.023882 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:17.043490 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6681296a-c668-4282-a398-d0b91e71976a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:17.044093 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6681296a-c668-4282-a398-d0b91e71976a None admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:17.044386 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6681296a-c668-4282-a398-d0b91e71976a None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:17.044672 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6681296a-c668-4282-a398-d0b91e71976a None admin] PATH_INFO: `/v3/users` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:17.045323 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6681296a-c668-4282-a398-d0b91e71976a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:17.046747 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6681296a-c668-4282-a398-d0b91e71976a None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:17.169054 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 28/61] 10.222.0.55 () {60 vars in 1132 bytes} [Wed Aug 24 08:33:16 2022] POST /identity/v3/users => generated 275 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:33:20.168594 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c96ed9f-d429-4100-8f02-fd27e6bdc3c5 None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:20.169607 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c96ed9f-d429-4100-8f02-fd27e6bdc3c5 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:20.170104 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c96ed9f-d429-4100-8f02-fd27e6bdc3c5 None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:20.171641 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 34/62] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:33:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:33:20.177179 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac260f38-8a55-4817-9923-a7c2dc2905f6 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:20.178078 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac260f38-8a55-4817-9923-a7c2dc2905f6 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:20.178566 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac260f38-8a55-4817-9923-a7c2dc2905f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:20.202724 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-ac260f38-8a55-4817-9923-a7c2dc2905f6 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:33:20.202724 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-ac260f38-8a55-4817-9923-a7c2dc2905f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:20.309979 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 29/63] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:33:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 08:33:20.319791 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a00b2ba-cd1c-4ca1-88dc-89c3341bfb13 None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:20.320154 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a00b2ba-cd1c-4ca1-88dc-89c3341bfb13 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:20.320373 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a00b2ba-cd1c-4ca1-88dc-89c3341bfb13 None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:20.321997 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 35/64] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:33:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:33:20.335049 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207344fb-e780-4d03-83e0-f30c15b64131 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:20.335392 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207344fb-e780-4d03-83e0-f30c15b64131 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:20.335607 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207344fb-e780-4d03-83e0-f30c15b64131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:20.367098 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-207344fb-e780-4d03-83e0-f30c15b64131 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:33:20.368365 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-207344fb-e780-4d03-83e0-f30c15b64131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:20.477518 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 30/65] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:33:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:33:20.603585 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a00b2ba-cd1c-4ca1-88dc-89c3341bfb13 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:20.617136 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:20.738531 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6d4705-21c8-40a3-974e-c552656f32c6 None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:20.738833 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6d4705-21c8-40a3-974e-c552656f32c6 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:20.738997 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6d4705-21c8-40a3-974e-c552656f32c6 None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:20.740339 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 31/66] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:33:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:33:20.745587 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a00b2ba-cd1c-4ca1-88dc-89c3341bfb13 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:20.756215 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c653ce-2b19-49b6-9387-d399b70082b8 None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:20.756500 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c653ce-2b19-49b6-9387-d399b70082b8 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:20.756809 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c653ce-2b19-49b6-9387-d399b70082b8 None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:20.758813 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 32/67] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:33:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:33:20.759697 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80402b1-4a64-44db-b99e-043250744de4 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:20.759979 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80402b1-4a64-44db-b99e-043250744de4 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:20.760252 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80402b1-4a64-44db-b99e-043250744de4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:20.779436 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-b80402b1-4a64-44db-b99e-043250744de4 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:33:20.780759 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-b80402b1-4a64-44db-b99e-043250744de4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:20.790261 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869593f6-2965-4769-b3e4-a91cec709c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:20.791676 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869593f6-2965-4769-b3e4-a91cec709c16 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:20.792170 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869593f6-2965-4769-b3e4-a91cec709c16 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:20.793078 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869593f6-2965-4769-b3e4-a91cec709c16 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:20.806373 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869593f6-2965-4769-b3e4-a91cec709c16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:33:20.807797 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:20.811385 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-869593f6-2965-4769-b3e4-a91cec709c16 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:33:20.811385 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:33:20.811385 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:33:20.811385 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:33:20.811385 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:33:20.811385 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:33:20.811385 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:33:20.811385 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:33:20.811385 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:33:20.811385 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:33:20.811385 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:33:20.811385 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:33:20.811385 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:33:20.811385 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:33:20.811385 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:33:20.811385 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:33:20.812249 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:33:20.812249 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:33:20.812249 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:33:20.812249 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:33:20.812249 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:33:20.812249 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:33:20.812249 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:33:20.812249 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:33:20.812249 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:33:20.812249 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:33:20.812249 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:33:20.812249 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:33:20.812249 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:33:20.812249 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:33:20.812249 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:33:20.812249 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:33:20.813049 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:33:20.813049 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:33:20.813049 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:33:20.813049 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:33:20.813049 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:33:20.813049 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:33:20.813049 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:33:20.813049 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:33:20.813049 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:33:20.813049 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:33:20.813049 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:33:20.813049 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:33:20.813049 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 36/68] 10.222.0.55 () {56 vars in 1096 bytes} [Wed Aug 24 08:33:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:33:20.814435 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af7a0c5-9318-4fb0-9f18-06313a33a640 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:20.814435 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af7a0c5-9318-4fb0-9f18-06313a33a640 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:20.814435 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af7a0c5-9318-4fb0-9f18-06313a33a640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:20.835126 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-5af7a0c5-9318-4fb0-9f18-06313a33a640 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:33:20.837013 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-5af7a0c5-9318-4fb0-9f18-06313a33a640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:20.870687 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 33/69] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:33:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:33:20.871553 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80402b1-4a64-44db-b99e-043250744de4 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:20.912807 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:20.918107 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 37/70] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:33:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:33:20.918972 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af7a0c5-9318-4fb0-9f18-06313a33a640 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:21.003778 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80402b1-4a64-44db-b99e-043250744de4 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:21.030107 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af7a0c5-9318-4fb0-9f18-06313a33a640 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:21.051078 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d503cd4-8913-4f2a-9dd2-5050567f6cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:21.052288 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d503cd4-8913-4f2a-9dd2-5050567f6cea None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:21.052596 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d503cd4-8913-4f2a-9dd2-5050567f6cea None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:21.052875 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d503cd4-8913-4f2a-9dd2-5050567f6cea None admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:21.053829 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d503cd4-8913-4f2a-9dd2-5050567f6cea None admin] RBAC: Adding query filter params (name=admin) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:33:21.054135 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d503cd4-8913-4f2a-9dd2-5050567f6cea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:21.056776 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d503cd4-8913-4f2a-9dd2-5050567f6cea None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:21.068217 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fcaed4-59fe-4a24-97c6-2421a89ef6eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:21.069068 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fcaed4-59fe-4a24-97c6-2421a89ef6eb None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:21.069432 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fcaed4-59fe-4a24-97c6-2421a89ef6eb None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:21.069783 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fcaed4-59fe-4a24-97c6-2421a89ef6eb None admin] PATH_INFO: `/v3/roles/member` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:21.073037 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 34/71] 10.222.0.55 () {56 vars in 1099 bytes} [Wed Aug 24 08:33:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:21.074739 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc9d2f0-0476-423e-a2b8-5ee391144c60 None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:21.075026 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc9d2f0-0476-423e-a2b8-5ee391144c60 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:21.075302 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc9d2f0-0476-423e-a2b8-5ee391144c60 None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:21.076682 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 35/72] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:33:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:33:21.079842 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc9d2f0-0476-423e-a2b8-5ee391144c60 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:21.087725 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fcaed4-59fe-4a24-97c6-2421a89ef6eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:33:21.093468 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-b1fcaed4-59fe-4a24-97c6-2421a89ef6eb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:33:21.093468 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:33:21.093468 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:33:21.093468 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:33:21.093468 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:33:21.093468 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:33:21.093468 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:33:21.093468 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:33:21.093468 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:33:21.093468 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:33:21.093468 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:33:21.093468 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:33:21.093468 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:33:21.093468 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:33:21.093468 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:33:21.093468 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:33:21.094555 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:33:21.094555 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:33:21.094555 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:33:21.094555 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:33:21.094555 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:33:21.094555 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:33:21.094555 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:33:21.094555 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:33:21.094555 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:33:21.094555 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:33:21.094555 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:33:21.094555 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:33:21.094555 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:33:21.094555 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:33:21.094555 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:33:21.094555 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:33:21.096469 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:33:21.096469 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:33:21.096469 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:33:21.096469 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:33:21.096469 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:33:21.096469 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:33:21.096469 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:33:21.096469 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:33:21.096469 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:33:21.096469 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:33:21.096469 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:33:21.096469 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:33:21.096469 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 38/73] 10.222.0.55 () {56 vars in 1099 bytes} [Wed Aug 24 08:33:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:33:21.096469 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465db2bf-8274-44c8-91e9-99245d19878b None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:21.098204 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465db2bf-8274-44c8-91e9-99245d19878b None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:21.098204 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465db2bf-8274-44c8-91e9-99245d19878b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:21.104078 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:21.121407 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-465db2bf-8274-44c8-91e9-99245d19878b None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:33:21.122626 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-465db2bf-8274-44c8-91e9-99245d19878b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:21.188439 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:21.202245 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc9d2f0-0476-423e-a2b8-5ee391144c60 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:21.210731 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 39/74] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:33:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:33:21.210731 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465db2bf-8274-44c8-91e9-99245d19878b None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:21.241003 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95e4273-cb4b-4583-8e7c-8b396dbb5339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:21.241640 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95e4273-cb4b-4583-8e7c-8b396dbb5339 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:21.241921 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95e4273-cb4b-4583-8e7c-8b396dbb5339 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:21.242701 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95e4273-cb4b-4583-8e7c-8b396dbb5339 None admin] PATH_INFO: `/v3/users/28800687066c4de989fe1940c9a30412` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:21.243849 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95e4273-cb4b-4583-8e7c-8b396dbb5339 None admin] RBAC: Authorizing `identity:get_user(user_id=28800687066c4de989fe1940c9a30412)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:21.245335 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95e4273-cb4b-4583-8e7c-8b396dbb5339 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:21.247193 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 36/75] 10.222.0.55 () {56 vars in 1177 bytes} [Wed Aug 24 08:33:21 2022] GET /identity/v3/users/28800687066c4de989fe1940c9a30412 => generated 247 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:33:21.248017 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95e4273-cb4b-4583-8e7c-8b396dbb5339 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:21.325139 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465db2bf-8274-44c8-91e9-99245d19878b None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:21.339714 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:21.352702 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95e4273-cb4b-4583-8e7c-8b396dbb5339 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:21.363531 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b937d926-7d09-49b6-abb2-9d2c172f639f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:21.364149 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b937d926-7d09-49b6-abb2-9d2c172f639f None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:21.364431 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b937d926-7d09-49b6-abb2-9d2c172f639f None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:21.364737 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b937d926-7d09-49b6-abb2-9d2c172f639f None admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:21.365425 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b937d926-7d09-49b6-abb2-9d2c172f639f None admin] RBAC: Adding query filter params (name=member) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:33:21.365727 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b937d926-7d09-49b6-abb2-9d2c172f639f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:21.367168 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b937d926-7d09-49b6-abb2-9d2c172f639f None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:21.381009 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 40/76] 10.222.0.55 () {56 vars in 1101 bytes} [Wed Aug 24 08:33:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:21.381009 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b937d926-7d09-49b6-abb2-9d2c172f639f None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:21.394707 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd9139e-79ab-4c5b-bac7-c6f9db6a59d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:21.395353 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd9139e-79ab-4c5b-bac7-c6f9db6a59d7 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:21.395635 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd9139e-79ab-4c5b-bac7-c6f9db6a59d7 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:21.395909 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd9139e-79ab-4c5b-bac7-c6f9db6a59d7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:21.406849 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd9139e-79ab-4c5b-bac7-c6f9db6a59d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:33:21.411591 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-ebd9139e-79ab-4c5b-bac7-c6f9db6a59d7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 08:33:21.411591 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:33:21.411591 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:33:21.411591 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:33:21.411591 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:33:21.411591 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:33:21.411591 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:33:21.411591 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:33:21.411591 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:33:21.411591 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:33:21.411591 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:33:21.411591 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:33:21.411591 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:33:21.411591 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:33:21.411591 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:33:21.411591 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:33:21.412483 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:33:21.412483 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:33:21.412483 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:33:21.412483 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:33:21.412483 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:33:21.412483 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:33:21.412483 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:33:21.412483 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:33:21.412483 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:33:21.412483 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:33:21.412483 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:33:21.412483 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:33:21.412483 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:33:21.412483 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:33:21.412483 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:33:21.412483 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:33:21.417530 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:33:21.417530 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:33:21.417530 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:33:21.417530 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:33:21.417530 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:33:21.417530 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:33:21.417530 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:33:21.417530 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:33:21.417530 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:33:21.417530 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:33:21.417530 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 08:33:21.417530 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:33:21.417530 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 37/77] 10.222.0.55 () {56 vars in 1114 bytes} [Wed Aug 24 08:33:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:33:21.417530 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd9139e-79ab-4c5b-bac7-c6f9db6a59d7 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:21.444533 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:21.494098 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b937d926-7d09-49b6-abb2-9d2c172f639f None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:21.523987 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd9139e-79ab-4c5b-bac7-c6f9db6a59d7 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:21.528333 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a2b488-01a6-4244-8f0f-d787e5b4f49d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:21.529022 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a2b488-01a6-4244-8f0f-d787e5b4f49d None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:21.529308 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a2b488-01a6-4244-8f0f-d787e5b4f49d None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:21.529587 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a2b488-01a6-4244-8f0f-d787e5b4f49d None admin] PATH_INFO: `/v3/domains/default` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:21.530730 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a2b488-01a6-4244-8f0f-d787e5b4f49d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:21.532174 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a2b488-01a6-4244-8f0f-d787e5b4f49d None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:21.533791 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 41/78] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:33:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:21.533791 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a2b488-01a6-4244-8f0f-d787e5b4f49d None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:21.564278 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4994354c-eb57-47d3-b707-746485d2079c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:21.564943 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4994354c-eb57-47d3-b707-746485d2079c None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:21.565260 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4994354c-eb57-47d3-b707-746485d2079c None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:21.565492 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4994354c-eb57-47d3-b707-746485d2079c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:21.583732 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:21.636775 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4994354c-eb57-47d3-b707-746485d2079c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:21.638439 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4994354c-eb57-47d3-b707-746485d2079c None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:21.639214 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a2b488-01a6-4244-8f0f-d787e5b4f49d None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:21.660024 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:21.674226 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205ecca2-fca3-4372-af41-183c9c80e328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:21.674869 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205ecca2-fca3-4372-af41-183c9c80e328 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:21.675152 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205ecca2-fca3-4372-af41-183c9c80e328 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:21.675432 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205ecca2-fca3-4372-af41-183c9c80e328 None admin] PATH_INFO: `/v3/users/07914530ee0141de9cc889c157da5a4c` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:21.692195 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205ecca2-fca3-4372-af41-183c9c80e328 None admin] RBAC: Authorizing `identity:get_user(user_id=07914530ee0141de9cc889c157da5a4c)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:21.693714 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205ecca2-fca3-4372-af41-183c9c80e328 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:21.695390 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 42/79] 10.222.0.55 () {56 vars in 1177 bytes} [Wed Aug 24 08:33:21 2022] GET /identity/v3/users/07914530ee0141de9cc889c157da5a4c => generated 275 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:21.696172 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205ecca2-fca3-4372-af41-183c9c80e328 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:21.742705 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 38/80] 10.222.0.55 () {58 vars in 1282 bytes} [Wed Aug 24 08:33:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:33:21.743612 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4994354c-eb57-47d3-b707-746485d2079c None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:21.794909 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:21.802746 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205ecca2-fca3-4372-af41-183c9c80e328 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:21.838204 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf55871-de65-45cc-b5f3-0e800212fbbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:21.838722 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf55871-de65-45cc-b5f3-0e800212fbbc None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:21.839005 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf55871-de65-45cc-b5f3-0e800212fbbc None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:21.839279 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf55871-de65-45cc-b5f3-0e800212fbbc None admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:21.839953 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf55871-de65-45cc-b5f3-0e800212fbbc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:33:21.840260 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf55871-de65-45cc-b5f3-0e800212fbbc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:21.841688 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf55871-de65-45cc-b5f3-0e800212fbbc None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:21.847606 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4994354c-eb57-47d3-b707-746485d2079c None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:21.850676 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 43/81] 10.222.0.55 () {56 vars in 1111 bytes} [Wed Aug 24 08:33:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:21.851267 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf55871-de65-45cc-b5f3-0e800212fbbc None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:21.887484 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505a5d4f-8bd1-452f-a7ba-5a28d6644d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:21.888128 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505a5d4f-8bd1-452f-a7ba-5a28d6644d50 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:21.888440 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505a5d4f-8bd1-452f-a7ba-5a28d6644d50 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:21.888771 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505a5d4f-8bd1-452f-a7ba-5a28d6644d50 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:21.894326 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:21.899954 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505a5d4f-8bd1-452f-a7ba-5a28d6644d50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:33:21.904855 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-505a5d4f-8bd1-452f-a7ba-5a28d6644d50 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:33:21.904855 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:33:21.904855 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:33:21.904855 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:33:21.904855 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:33:21.904855 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:33:21.904855 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:33:21.904855 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:33:21.904855 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:33:21.904855 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:33:21.904855 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:33:21.904855 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:33:21.904855 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:33:21.904855 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:33:21.904855 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:33:21.904855 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:33:21.908980 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:33:21.908980 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:33:21.908980 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:33:21.908980 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:33:21.908980 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:33:21.908980 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:33:21.908980 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:33:21.908980 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:33:21.908980 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:33:21.908980 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:33:21.908980 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:33:21.908980 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:33:21.908980 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:33:21.908980 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:33:21.908980 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:33:21.908980 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:33:21.909934 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:33:21.909934 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:33:21.909934 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:33:21.909934 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:33:21.909934 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:33:21.909934 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:33:21.909934 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:33:21.909934 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:33:21.909934 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:33:21.909934 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:33:21.909934 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:33:21.909934 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:33:21.909934 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 39/82] 10.222.0.55 () {56 vars in 1099 bytes} [Wed Aug 24 08:33:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:33:21.909934 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505a5d4f-8bd1-452f-a7ba-5a28d6644d50 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:21.957062 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf55871-de65-45cc-b5f3-0e800212fbbc None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:21.991837 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1689a13-e851-4fff-8498-817f7eea8713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:21.992456 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1689a13-e851-4fff-8498-817f7eea8713 None admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:21.992770 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1689a13-e851-4fff-8498-817f7eea8713 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:21.993048 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1689a13-e851-4fff-8498-817f7eea8713 None admin] PATH_INFO: `/v3/users` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:21.993708 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1689a13-e851-4fff-8498-817f7eea8713 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:21.995169 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1689a13-e851-4fff-8498-817f7eea8713 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:22.009185 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505a5d4f-8bd1-452f-a7ba-5a28d6644d50 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:22.018695 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:22.044832 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21f9199-5e92-4eec-a9c6-f8501bde4a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:22.045473 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21f9199-5e92-4eec-a9c6-f8501bde4a93 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:22.045756 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21f9199-5e92-4eec-a9c6-f8501bde4a93 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:22.046034 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21f9199-5e92-4eec-a9c6-f8501bde4a93 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:22.081874 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 44/83] 10.222.0.55 () {60 vars in 1132 bytes} [Wed Aug 24 08:33:21 2022] POST /identity/v3/users => generated 289 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:33:22.090191 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1689a13-e851-4fff-8498-817f7eea8713 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:22.129769 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21f9199-5e92-4eec-a9c6-f8501bde4a93 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:22.131647 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21f9199-5e92-4eec-a9c6-f8501bde4a93 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:22.207162 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:22.215602 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1689a13-e851-4fff-8498-817f7eea8713 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:22.257832 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0655168-c798-4e82-a881-106b32aae0f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:22.258981 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0655168-c798-4e82-a881-106b32aae0f9 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:22.259268 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0655168-c798-4e82-a881-106b32aae0f9 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:22.260200 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 40/84] 10.222.0.55 () {58 vars in 1282 bytes} [Wed Aug 24 08:33:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:33:22.260676 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0655168-c798-4e82-a881-106b32aae0f9 None admin] PATH_INFO: `/v3/projects/f8b7cfbd340b4753bd898ddc0a462a03` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:22.261461 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21f9199-5e92-4eec-a9c6-f8501bde4a93 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:22.265652 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0655168-c798-4e82-a881-106b32aae0f9 None admin] RBAC: Authorizing `identity:get_project(project_id=f8b7cfbd340b4753bd898ddc0a462a03)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:22.267612 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0655168-c798-4e82-a881-106b32aae0f9 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:22.269939 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 45/85] 10.222.0.55 () {56 vars in 1186 bytes} [Wed Aug 24 08:33:22 2022] GET /identity/v3/projects/f8b7cfbd340b4753bd898ddc0a462a03 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:22.269939 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0655168-c798-4e82-a881-106b32aae0f9 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:22.272601 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:22.397450 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21f9199-5e92-4eec-a9c6-f8501bde4a93 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:22.401974 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0655168-c798-4e82-a881-106b32aae0f9 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:22.437480 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f942354-fe03-4abb-ac89-45400e7a448a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:22.438774 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f942354-fe03-4abb-ac89-45400e7a448a None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:22.438774 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f942354-fe03-4abb-ac89-45400e7a448a None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:22.438774 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f942354-fe03-4abb-ac89-45400e7a448a None admin] PATH_INFO: `/v3/users/07914530ee0141de9cc889c157da5a4c` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:22.439951 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f942354-fe03-4abb-ac89-45400e7a448a None admin] RBAC: Authorizing `identity:get_user(user_id=07914530ee0141de9cc889c157da5a4c)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:22.441519 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f942354-fe03-4abb-ac89-45400e7a448a None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:22.444023 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 41/86] 10.222.0.55 () {56 vars in 1177 bytes} [Wed Aug 24 08:33:22 2022] GET /identity/v3/users/07914530ee0141de9cc889c157da5a4c => generated 275 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:22.444023 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f942354-fe03-4abb-ac89-45400e7a448a None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:22.450899 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c38ab4-9491-48dd-930d-a96b251d601e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:22.451537 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c38ab4-9491-48dd-930d-a96b251d601e None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:22.451819 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c38ab4-9491-48dd-930d-a96b251d601e None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:22.452095 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c38ab4-9491-48dd-930d-a96b251d601e None admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:22.452809 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c38ab4-9491-48dd-930d-a96b251d601e None admin] RBAC: Adding query filter params (name=member) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:33:22.453112 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c38ab4-9491-48dd-930d-a96b251d601e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:22.455056 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c38ab4-9491-48dd-930d-a96b251d601e None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:22.467058 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 46/87] 10.222.0.55 () {56 vars in 1101 bytes} [Wed Aug 24 08:33:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:22.468494 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c38ab4-9491-48dd-930d-a96b251d601e None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:22.506610 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:22.537441 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:22.567814 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f942354-fe03-4abb-ac89-45400e7a448a None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:22.594346 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c38ab4-9491-48dd-930d-a96b251d601e None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:22.603825 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12543bd2-260a-40e2-9462-34dfba4101f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:22.604570 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12543bd2-260a-40e2-9462-34dfba4101f6 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:22.604920 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12543bd2-260a-40e2-9462-34dfba4101f6 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:22.605261 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12543bd2-260a-40e2-9462-34dfba4101f6 None admin] PATH_INFO: `/v3/projects/f8b7cfbd340b4753bd898ddc0a462a03` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:22.606503 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12543bd2-260a-40e2-9462-34dfba4101f6 None admin] RBAC: Authorizing `identity:get_project(project_id=f8b7cfbd340b4753bd898ddc0a462a03)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:22.608016 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12543bd2-260a-40e2-9462-34dfba4101f6 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:22.614786 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 42/88] 10.222.0.55 () {56 vars in 1186 bytes} [Wed Aug 24 08:33:22 2022] GET /identity/v3/projects/f8b7cfbd340b4753bd898ddc0a462a03 => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:22.615793 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12543bd2-260a-40e2-9462-34dfba4101f6 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:22.636405 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cced43e5-8ddc-4117-b1c1-de96347f167c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:22.637206 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cced43e5-8ddc-4117-b1c1-de96347f167c None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:22.637596 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cced43e5-8ddc-4117-b1c1-de96347f167c None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:22.637972 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cced43e5-8ddc-4117-b1c1-de96347f167c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:22.638824 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cced43e5-8ddc-4117-b1c1-de96347f167c None admin] RBAC: Adding query filter params (role.id=b870cd1acaa744e1ae1d58b9ac283821, scope.project.id=f8b7cfbd340b4753bd898ddc0a462a03, user.id=28800687066c4de989fe1940c9a30412) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:33:22.639286 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cced43e5-8ddc-4117-b1c1-de96347f167c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:22.640966 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cced43e5-8ddc-4117-b1c1-de96347f167c None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:22.647533 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 47/89] 10.222.0.55 () {56 vars in 1374 bytes} [Wed Aug 24 08:33:22 2022] GET /identity/v3/role_assignments?user.id=28800687066c4de989fe1940c9a30412&scope.project.id=f8b7cfbd340b4753bd898ddc0a462a03&role.id=b870cd1acaa744e1ae1d58b9ac283821 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:22.648410 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cced43e5-8ddc-4117-b1c1-de96347f167c None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:22.771110 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:22.784301 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cced43e5-8ddc-4117-b1c1-de96347f167c None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:22.826245 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12543bd2-260a-40e2-9462-34dfba4101f6 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:22.833200 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a0aa48-0e9a-4b23-86c4-ea45d0494381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:22.833858 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a0aa48-0e9a-4b23-86c4-ea45d0494381 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:22.834147 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a0aa48-0e9a-4b23-86c4-ea45d0494381 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:22.834432 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a0aa48-0e9a-4b23-86c4-ea45d0494381 None admin] PATH_INFO: `/v3/users/07914530ee0141de9cc889c157da5a4c` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:22.835599 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a0aa48-0e9a-4b23-86c4-ea45d0494381 None admin] RBAC: Authorizing `identity:get_user(user_id=07914530ee0141de9cc889c157da5a4c)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:22.837657 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a0aa48-0e9a-4b23-86c4-ea45d0494381 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:22.840137 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 48/90] 10.222.0.55 () {56 vars in 1177 bytes} [Wed Aug 24 08:33:22 2022] GET /identity/v3/users/07914530ee0141de9cc889c157da5a4c => generated 275 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:22.844708 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a0aa48-0e9a-4b23-86c4-ea45d0494381 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:22.901385 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d79fbe8-48f2-4a5c-8d4a-c2414a272e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:22.902359 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d79fbe8-48f2-4a5c-8d4a-c2414a272e05 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:22.902840 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d79fbe8-48f2-4a5c-8d4a-c2414a272e05 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:22.903327 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d79fbe8-48f2-4a5c-8d4a-c2414a272e05 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:22.931452 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:22.968110 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a0aa48-0e9a-4b23-86c4-ea45d0494381 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:23.013933 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdba534-5af6-43c1-973e-daae94eb4a75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:23.014575 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdba534-5af6-43c1-973e-daae94eb4a75 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:23.014858 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdba534-5af6-43c1-973e-daae94eb4a75 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:23.015130 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdba534-5af6-43c1-973e-daae94eb4a75 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:23.016418 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdba534-5af6-43c1-973e-daae94eb4a75 None admin] RBAC: Adding query filter params (role.id=b674fd0a592e4e0cada811bba5ceea4d, scope.project.id=f8b7cfbd340b4753bd898ddc0a462a03, user.id=07914530ee0141de9cc889c157da5a4c) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:33:23.016764 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdba534-5af6-43c1-973e-daae94eb4a75 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:23.018397 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdba534-5af6-43c1-973e-daae94eb4a75 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:23.026319 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 49/91] 10.222.0.55 () {56 vars in 1374 bytes} [Wed Aug 24 08:33:22 2022] GET /identity/v3/role_assignments?user.id=07914530ee0141de9cc889c157da5a4c&scope.project.id=f8b7cfbd340b4753bd898ddc0a462a03&role.id=b674fd0a592e4e0cada811bba5ceea4d => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:23.027358 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdba534-5af6-43c1-973e-daae94eb4a75 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:23.059630 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:23.103104 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d79fbe8-48f2-4a5c-8d4a-c2414a272e05 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:23.109406 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d79fbe8-48f2-4a5c-8d4a-c2414a272e05 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:23.156000 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdba534-5af6-43c1-973e-daae94eb4a75 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:23.208499 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d710821-21eb-4cb5-93d5-8897db0a8bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:23.209307 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d710821-21eb-4cb5-93d5-8897db0a8bd2 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:23.209479 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d710821-21eb-4cb5-93d5-8897db0a8bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:23.209726 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d710821-21eb-4cb5-93d5-8897db0a8bd2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:23.306761 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 43/92] 10.222.0.55 () {58 vars in 1282 bytes} [Wed Aug 24 08:33:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 691 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:33:23.313057 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d79fbe8-48f2-4a5c-8d4a-c2414a272e05 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:23.357181 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d710821-21eb-4cb5-93d5-8897db0a8bd2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:23.359009 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d710821-21eb-4cb5-93d5-8897db0a8bd2 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:23.411279 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:23.414135 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:23.537948 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d79fbe8-48f2-4a5c-8d4a-c2414a272e05 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:23.560929 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 50/93] 10.222.0.55 () {58 vars in 1282 bytes} [Wed Aug 24 08:33:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 534 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:33:23.567248 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d710821-21eb-4cb5-93d5-8897db0a8bd2 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:23.606606 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418c0b38-0a82-42c0-ab85-013c6083e039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:23.607261 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418c0b38-0a82-42c0-ab85-013c6083e039 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:23.607548 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418c0b38-0a82-42c0-ab85-013c6083e039 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:23.607826 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418c0b38-0a82-42c0-ab85-013c6083e039 None admin] PATH_INFO: `/v3/projects/f8b7cfbd340b4753bd898ddc0a462a03` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:23.612668 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418c0b38-0a82-42c0-ab85-013c6083e039 None admin] RBAC: Authorizing `identity:get_project(project_id=f8b7cfbd340b4753bd898ddc0a462a03)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:23.614152 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418c0b38-0a82-42c0-ab85-013c6083e039 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:23.616476 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 44/94] 10.222.0.55 () {56 vars in 1186 bytes} [Wed Aug 24 08:33:23 2022] GET /identity/v3/projects/f8b7cfbd340b4753bd898ddc0a462a03 => generated 298 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:23.623227 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418c0b38-0a82-42c0-ab85-013c6083e039 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:23.681099 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d710821-21eb-4cb5-93d5-8897db0a8bd2 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:23.707600 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:23.721064 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f602368-892c-49bb-8290-0639d992ab3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:23.722062 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f602368-892c-49bb-8290-0639d992ab3e None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:23.723610 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f602368-892c-49bb-8290-0639d992ab3e None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:23.723610 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f602368-892c-49bb-8290-0639d992ab3e None admin] PATH_INFO: `/v3/projects/abc83224889e4e0f8d1675a2cbc7657b` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:23.724512 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f602368-892c-49bb-8290-0639d992ab3e None admin] RBAC: Authorizing `identity:get_project(project_id=abc83224889e4e0f8d1675a2cbc7657b)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:23.726533 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f602368-892c-49bb-8290-0639d992ab3e None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:23.729389 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 51/95] 10.222.0.55 () {56 vars in 1186 bytes} [Wed Aug 24 08:33:23 2022] GET /identity/v3/projects/abc83224889e4e0f8d1675a2cbc7657b => generated 312 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:23.735618 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f602368-892c-49bb-8290-0639d992ab3e None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:23.749699 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418c0b38-0a82-42c0-ab85-013c6083e039 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:23.765546 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:23.789938 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a77910-7704-460f-a051-46fc016a01fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:23.790570 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a77910-7704-460f-a051-46fc016a01fc None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:23.790849 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a77910-7704-460f-a051-46fc016a01fc None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:23.791125 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a77910-7704-460f-a051-46fc016a01fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:23.791928 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a77910-7704-460f-a051-46fc016a01fc None admin] RBAC: Adding query filter params (role.id=8b5ad96041894155b0138a676ead607e, scope.project.id=f8b7cfbd340b4753bd898ddc0a462a03, user.id=07914530ee0141de9cc889c157da5a4c) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:33:23.792239 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a77910-7704-460f-a051-46fc016a01fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:23.794307 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a77910-7704-460f-a051-46fc016a01fc None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:23.803878 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 45/96] 10.222.0.55 () {56 vars in 1374 bytes} [Wed Aug 24 08:33:23 2022] GET /identity/v3/role_assignments?user.id=07914530ee0141de9cc889c157da5a4c&scope.project.id=f8b7cfbd340b4753bd898ddc0a462a03&role.id=8b5ad96041894155b0138a676ead607e => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:33:23.857602 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f602368-892c-49bb-8290-0639d992ab3e None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:23.897219 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfa84a1-6200-4766-bfaa-5b9ec6f7c81f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:23.898186 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfa84a1-6200-4766-bfaa-5b9ec6f7c81f None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:23.898706 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfa84a1-6200-4766-bfaa-5b9ec6f7c81f None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:23.899203 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfa84a1-6200-4766-bfaa-5b9ec6f7c81f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:23.900175 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfa84a1-6200-4766-bfaa-5b9ec6f7c81f None admin] RBAC: Adding query filter params (role.id=b674fd0a592e4e0cada811bba5ceea4d, scope.project.id=abc83224889e4e0f8d1675a2cbc7657b, user.id=07914530ee0141de9cc889c157da5a4c) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:33:23.900718 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfa84a1-6200-4766-bfaa-5b9ec6f7c81f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:23.902455 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfa84a1-6200-4766-bfaa-5b9ec6f7c81f None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:23.909219 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 52/97] 10.222.0.55 () {56 vars in 1374 bytes} [Wed Aug 24 08:33:23 2022] GET /identity/v3/role_assignments?user.id=07914530ee0141de9cc889c157da5a4c&scope.project.id=abc83224889e4e0f8d1675a2cbc7657b&role.id=b674fd0a592e4e0cada811bba5ceea4d => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:33:24.684314 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939a81e8-fd50-4342-8dd6-32da9e445678 None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:24.684855 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939a81e8-fd50-4342-8dd6-32da9e445678 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:24.684931 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939a81e8-fd50-4342-8dd6-32da9e445678 None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:24.686258 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 46/98] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:33:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:33:24.692179 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4f6cc3-5f27-4ef8-9484-caf8b6056bd3 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:24.692668 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4f6cc3-5f27-4ef8-9484-caf8b6056bd3 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:24.692763 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4f6cc3-5f27-4ef8-9484-caf8b6056bd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:24.713353 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-8b4f6cc3-5f27-4ef8-9484-caf8b6056bd3 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:33:24.714593 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-8b4f6cc3-5f27-4ef8-9484-caf8b6056bd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:24.788814 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:24.814558 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 53/99] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:33:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:33:24.850593 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b210948b-d9bb-4516-a639-3b2edaf94f8b None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:24.850890 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b210948b-d9bb-4516-a639-3b2edaf94f8b None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:24.851161 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b210948b-d9bb-4516-a639-3b2edaf94f8b None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:24.853378 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 47/100] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:33:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:33:24.859283 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a297b5f-8a4a-405f-8588-e6f5c0acc1f2 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:24.859614 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a297b5f-8a4a-405f-8588-e6f5c0acc1f2 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:24.859869 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a297b5f-8a4a-405f-8588-e6f5c0acc1f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:24.881687 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-4a297b5f-8a4a-405f-8588-e6f5c0acc1f2 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:33:24.882972 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-4a297b5f-8a4a-405f-8588-e6f5c0acc1f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:24.972306 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 54/101] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:33:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:33:25.189210 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b210948b-d9bb-4516-a639-3b2edaf94f8b None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:25.300436 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b210948b-d9bb-4516-a639-3b2edaf94f8b None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:25.339952 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f73b228-f468-424e-a8ec-f4b107b2d208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:25.340687 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f73b228-f468-424e-a8ec-f4b107b2d208 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:25.341338 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f73b228-f468-424e-a8ec-f4b107b2d208 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:25.341620 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f73b228-f468-424e-a8ec-f4b107b2d208 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:25.353539 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f73b228-f468-424e-a8ec-f4b107b2d208 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:33:25.358384 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-1f73b228-f468-424e-a8ec-f4b107b2d208 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 08:33:25.358384 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:33:25.358384 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:33:25.358384 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:33:25.358384 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:33:25.358384 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:33:25.358384 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:33:25.358384 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:33:25.358384 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:33:25.358384 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:33:25.358384 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:33:25.358384 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:33:25.358384 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:33:25.358384 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:33:25.358384 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:33:25.358384 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:33:25.359271 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:33:25.359271 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:33:25.359271 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:33:25.359271 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:33:25.359271 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:33:25.359271 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:33:25.359271 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:33:25.359271 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:33:25.359271 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:33:25.359271 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:33:25.359271 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:33:25.359271 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:33:25.359271 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:33:25.359271 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:33:25.359271 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:33:25.359271 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:33:25.360036 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:33:25.360036 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:33:25.360036 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:33:25.360036 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:33:25.360036 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:33:25.360036 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:33:25.360036 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:33:25.360036 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:33:25.360036 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:33:25.360036 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:33:25.360036 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 08:33:25.360036 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:33:25.360036 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 48/102] 10.222.0.55 () {56 vars in 1099 bytes} [Wed Aug 24 08:33:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:33:25.366860 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a297b5f-8a4a-405f-8588-e6f5c0acc1f2 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:25.481904 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86793871-5ef3-41a2-89aa-0fa7da5fb05d None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:25.483622 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86793871-5ef3-41a2-89aa-0fa7da5fb05d None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:25.483622 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86793871-5ef3-41a2-89aa-0fa7da5fb05d None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:25.483770 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 49/103] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:33:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:33:25.489181 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1593c11c-f533-4c8b-8873-eb77a85ee02d None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:25.489521 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1593c11c-f533-4c8b-8873-eb77a85ee02d None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:25.489804 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1593c11c-f533-4c8b-8873-eb77a85ee02d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:25.505945 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a297b5f-8a4a-405f-8588-e6f5c0acc1f2 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:25.522630 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:25.548698 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb4c7f5-4088-4e1e-bfc3-216ea2395443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:25.549379 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb4c7f5-4088-4e1e-bfc3-216ea2395443 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:25.549667 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb4c7f5-4088-4e1e-bfc3-216ea2395443 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:25.549978 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb4c7f5-4088-4e1e-bfc3-216ea2395443 None admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:25.556957 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb4c7f5-4088-4e1e-bfc3-216ea2395443 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:33:25.557544 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb4c7f5-4088-4e1e-bfc3-216ea2395443 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:25.563538 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb4c7f5-4088-4e1e-bfc3-216ea2395443 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:25.565701 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-1593c11c-f533-4c8b-8873-eb77a85ee02d None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:33:25.566975 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-1593c11c-f533-4c8b-8873-eb77a85ee02d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:25.582739 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 55/104] 10.222.0.55 () {56 vars in 1101 bytes} [Wed Aug 24 08:33:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:25.583422 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb4c7f5-4088-4e1e-bfc3-216ea2395443 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:25.650918 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:25.756450 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 50/105] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:33:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 08:33:25.757276 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1593c11c-f533-4c8b-8873-eb77a85ee02d None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:25.762397 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb4c7f5-4088-4e1e-bfc3-216ea2395443 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:25.846578 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ca0cd4-f492-4793-a448-889264f6db60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:25.847148 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ca0cd4-f492-4793-a448-889264f6db60 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:25.847431 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ca0cd4-f492-4793-a448-889264f6db60 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:25.847711 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ca0cd4-f492-4793-a448-889264f6db60 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:25.850078 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ca0cd4-f492-4793-a448-889264f6db60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:25.851567 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ca0cd4-f492-4793-a448-889264f6db60 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:25.858548 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 56/106] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:33:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:25.859926 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ca0cd4-f492-4793-a448-889264f6db60 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:25.913730 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:25.999169 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1593c11c-f533-4c8b-8873-eb77a85ee02d None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:25.999496 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ca0cd4-f492-4793-a448-889264f6db60 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:26.053942 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af87d152-357f-4240-a245-e9ac3240cf87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:26.054939 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af87d152-357f-4240-a245-e9ac3240cf87 None admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:26.056074 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af87d152-357f-4240-a245-e9ac3240cf87 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:26.056569 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af87d152-357f-4240-a245-e9ac3240cf87 None admin] PATH_INFO: `/v3/projects` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:26.057455 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af87d152-357f-4240-a245-e9ac3240cf87 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:26.059301 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af87d152-357f-4240-a245-e9ac3240cf87 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:26.090164 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452daab0-c354-4d03-9b0d-8f6ca592eca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:26.090828 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452daab0-c354-4d03-9b0d-8f6ca592eca2 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:26.091112 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452daab0-c354-4d03-9b0d-8f6ca592eca2 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:26.091386 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452daab0-c354-4d03-9b0d-8f6ca592eca2 None admin] PATH_INFO: `/v3/users/5b29b7c0626f4c3da644d63cf7510055` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:26.109175 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452daab0-c354-4d03-9b0d-8f6ca592eca2 None admin] RBAC: Authorizing `identity:get_user(user_id=5b29b7c0626f4c3da644d63cf7510055)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:26.110814 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452daab0-c354-4d03-9b0d-8f6ca592eca2 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:26.113017 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 51/107] 10.222.0.55 () {56 vars in 1177 bytes} [Wed Aug 24 08:33:25 2022] GET /identity/v3/users/5b29b7c0626f4c3da644d63cf7510055 => generated 289 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:26.120728 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452daab0-c354-4d03-9b0d-8f6ca592eca2 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:26.182724 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:26.187302 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 57/108] 10.222.0.55 () {60 vars in 1141 bytes} [Wed Aug 24 08:33:25 2022] POST /identity/v3/projects => generated 302 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:33:26.246406 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af87d152-357f-4240-a245-e9ac3240cf87 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:26.295099 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452daab0-c354-4d03-9b0d-8f6ca592eca2 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:26.336099 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5d46d2-b77a-464c-9f99-7c05d9642253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:26.337307 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5d46d2-b77a-464c-9f99-7c05d9642253 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:26.337592 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5d46d2-b77a-464c-9f99-7c05d9642253 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:26.337872 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5d46d2-b77a-464c-9f99-7c05d9642253 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:26.405932 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:26.420682 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5d46d2-b77a-464c-9f99-7c05d9642253 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:26.422261 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5d46d2-b77a-464c-9f99-7c05d9642253 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:26.422370 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af87d152-357f-4240-a245-e9ac3240cf87 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:26.455392 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:26.470314 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e40dadc-d38d-48af-9a57-7190316b1b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:26.471901 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e40dadc-d38d-48af-9a57-7190316b1b18 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:26.472184 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e40dadc-d38d-48af-9a57-7190316b1b18 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:26.472501 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e40dadc-d38d-48af-9a57-7190316b1b18 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:26.489012 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e40dadc-d38d-48af-9a57-7190316b1b18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:33:26.495401 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-5e40dadc-d38d-48af-9a57-7190316b1b18 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:33:26.495401 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:33:26.495401 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:33:26.495401 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:33:26.495401 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:33:26.495401 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:33:26.495401 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:33:26.495401 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:33:26.495401 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:33:26.495401 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:33:26.495401 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:33:26.495401 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:33:26.495401 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:33:26.495401 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:33:26.495401 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:33:26.495401 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:33:26.497814 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:33:26.497814 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:33:26.497814 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:33:26.497814 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:33:26.497814 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:33:26.497814 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:33:26.497814 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:33:26.497814 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:33:26.497814 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:33:26.497814 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:33:26.497814 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:33:26.497814 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:33:26.497814 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:33:26.497814 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:33:26.497814 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:33:26.497814 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:33:26.498589 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:33:26.498589 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:33:26.498589 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:33:26.498589 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:33:26.498589 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:33:26.498589 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:33:26.498589 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:33:26.498589 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:33:26.498589 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:33:26.498589 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:33:26.498589 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:33:26.498589 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:33:26.498589 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 58/109] 10.222.0.55 () {56 vars in 1096 bytes} [Wed Aug 24 08:33:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:33:26.502062 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406e60a4-5990-4bae-be82-bf3aaa3ff4f3 None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:26.502062 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406e60a4-5990-4bae-be82-bf3aaa3ff4f3 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:26.502062 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406e60a4-5990-4bae-be82-bf3aaa3ff4f3 None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:26.502545 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 59/110] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:33:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:33:26.505284 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406e60a4-5990-4bae-be82-bf3aaa3ff4f3 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:26.575987 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 52/111] 10.222.0.55 () {58 vars in 1282 bytes} [Wed Aug 24 08:33:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:33:26.577965 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402d530e-cec0-41bc-819f-36151b72a5b9 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:26.578340 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402d530e-cec0-41bc-819f-36151b72a5b9 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:26.579068 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402d530e-cec0-41bc-819f-36151b72a5b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:26.608619 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-402d530e-cec0-41bc-819f-36151b72a5b9 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:33:26.610224 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-402d530e-cec0-41bc-819f-36151b72a5b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:26.665436 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406e60a4-5990-4bae-be82-bf3aaa3ff4f3 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:26.692172 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 53/112] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:33:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:33:26.693661 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402d530e-cec0-41bc-819f-36151b72a5b9 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:26.729075 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210a4443-05ec-4bf0-b6e1-2218f8ba1daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:26.729701 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210a4443-05ec-4bf0-b6e1-2218f8ba1daa None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:26.729983 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210a4443-05ec-4bf0-b6e1-2218f8ba1daa None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:26.730281 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210a4443-05ec-4bf0-b6e1-2218f8ba1daa None admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:26.731029 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210a4443-05ec-4bf0-b6e1-2218f8ba1daa None admin] RBAC: Adding query filter params (name=admin) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:33:26.731352 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210a4443-05ec-4bf0-b6e1-2218f8ba1daa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:26.732829 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210a4443-05ec-4bf0-b6e1-2218f8ba1daa None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:26.742824 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 60/113] 10.222.0.55 () {56 vars in 1099 bytes} [Wed Aug 24 08:33:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:26.748518 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210a4443-05ec-4bf0-b6e1-2218f8ba1daa None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:26.762940 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:26.786070 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:26.817065 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402d530e-cec0-41bc-819f-36151b72a5b9 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:26.862258 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0ed908-d143-4498-8440-92cc7614d40c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:26.863321 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0ed908-d143-4498-8440-92cc7614d40c None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:26.863808 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0ed908-d143-4498-8440-92cc7614d40c None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:26.864406 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0ed908-d143-4498-8440-92cc7614d40c None admin] PATH_INFO: `/v3/projects/f8b7cfbd340b4753bd898ddc0a462a03` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:26.868860 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0ed908-d143-4498-8440-92cc7614d40c None admin] RBAC: Authorizing `identity:get_project(project_id=f8b7cfbd340b4753bd898ddc0a462a03)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:26.870797 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0ed908-d143-4498-8440-92cc7614d40c None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:26.873256 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 54/114] 10.222.0.55 () {56 vars in 1186 bytes} [Wed Aug 24 08:33:26 2022] GET /identity/v3/projects/f8b7cfbd340b4753bd898ddc0a462a03 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:26.879079 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0ed908-d143-4498-8440-92cc7614d40c None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:26.931580 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210a4443-05ec-4bf0-b6e1-2218f8ba1daa None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:26.993857 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cc5020-898e-4b7a-b3cf-48516d6bff26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:26.994764 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cc5020-898e-4b7a-b3cf-48516d6bff26 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:26.995197 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cc5020-898e-4b7a-b3cf-48516d6bff26 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:26.995620 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cc5020-898e-4b7a-b3cf-48516d6bff26 None admin] PATH_INFO: `/v3/users/28800687066c4de989fe1940c9a30412` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:26.997446 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cc5020-898e-4b7a-b3cf-48516d6bff26 None admin] RBAC: Authorizing `identity:get_user(user_id=28800687066c4de989fe1940c9a30412)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:26.999084 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cc5020-898e-4b7a-b3cf-48516d6bff26 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:27.000900 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 61/115] 10.222.0.55 () {56 vars in 1177 bytes} [Wed Aug 24 08:33:26 2022] GET /identity/v3/users/28800687066c4de989fe1940c9a30412 => generated 247 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:27.006783 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cc5020-898e-4b7a-b3cf-48516d6bff26 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:27.033234 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0ed908-d143-4498-8440-92cc7614d40c None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:27.060827 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:27.080990 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c6b285-345a-4444-b5cc-1a815515394f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:27.082108 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c6b285-345a-4444-b5cc-1a815515394f None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:27.082880 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c6b285-345a-4444-b5cc-1a815515394f None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:27.084457 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c6b285-345a-4444-b5cc-1a815515394f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:27.085438 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c6b285-345a-4444-b5cc-1a815515394f None admin] RBAC: Adding query filter params (role.id=e308031ba9a84b79bcdf501455ebe0d7, scope.project.id=f8b7cfbd340b4753bd898ddc0a462a03, user.id=5b29b7c0626f4c3da644d63cf7510055) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:33:27.086190 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c6b285-345a-4444-b5cc-1a815515394f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:27.090422 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c6b285-345a-4444-b5cc-1a815515394f None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:27.097491 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 55/116] 10.222.0.55 () {56 vars in 1374 bytes} [Wed Aug 24 08:33:26 2022] GET /identity/v3/role_assignments?user.id=5b29b7c0626f4c3da644d63cf7510055&scope.project.id=f8b7cfbd340b4753bd898ddc0a462a03&role.id=e308031ba9a84b79bcdf501455ebe0d7 => generated 259 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:33:27.099218 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fb758c-7bf1-46a4-86d3-ef6256fb6b36 None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:27.099724 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fb758c-7bf1-46a4-86d3-ef6256fb6b36 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:27.100165 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fb758c-7bf1-46a4-86d3-ef6256fb6b36 None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:27.101800 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 56/117] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:33:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:33:27.110601 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde390dd-810c-4b0d-ad29-e62793c9ac7e None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:27.111166 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde390dd-810c-4b0d-ad29-e62793c9ac7e None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:27.112013 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde390dd-810c-4b0d-ad29-e62793c9ac7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:27.152991 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-bde390dd-810c-4b0d-ad29-e62793c9ac7e None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:33:27.159967 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-bde390dd-810c-4b0d-ad29-e62793c9ac7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:27.192929 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:27.227351 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cc5020-898e-4b7a-b3cf-48516d6bff26 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:27.279276 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 57/118] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:33:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 08:33:27.281046 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db532e8-c225-4259-9766-fc30d4554b78 None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:27.281339 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db532e8-c225-4259-9766-fc30d4554b78 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:27.281840 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db532e8-c225-4259-9766-fc30d4554b78 None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:27.283878 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 58/119] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:33:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:33:27.284556 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db532e8-c225-4259-9766-fc30d4554b78 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:27.298783 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1918aa53-0478-4686-8fec-16836713c92a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:27.299996 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1918aa53-0478-4686-8fec-16836713c92a None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:27.300574 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1918aa53-0478-4686-8fec-16836713c92a None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:27.301119 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1918aa53-0478-4686-8fec-16836713c92a None admin] PATH_INFO: `/v3/projects/f8b7cfbd340b4753bd898ddc0a462a03` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:27.303444 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1918aa53-0478-4686-8fec-16836713c92a None admin] RBAC: Authorizing `identity:get_project(project_id=f8b7cfbd340b4753bd898ddc0a462a03)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:27.306830 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1918aa53-0478-4686-8fec-16836713c92a None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:27.310315 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 62/120] 10.222.0.55 () {56 vars in 1186 bytes} [Wed Aug 24 08:33:27 2022] GET /identity/v3/projects/f8b7cfbd340b4753bd898ddc0a462a03 => generated 298 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:33:27.315998 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c809aba2-41f0-4ed1-891f-b69920d570fb None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:27.315998 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c809aba2-41f0-4ed1-891f-b69920d570fb None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:27.315998 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c809aba2-41f0-4ed1-891f-b69920d570fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:27.339850 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-c809aba2-41f0-4ed1-891f-b69920d570fb None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:33:27.341386 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-c809aba2-41f0-4ed1-891f-b69920d570fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:27.428246 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 63/121] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:33:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:33:27.429692 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c809aba2-41f0-4ed1-891f-b69920d570fb None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:27.430124 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db532e8-c225-4259-9766-fc30d4554b78 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:27.454415 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:27.483367 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f3ea97-a7f2-43d4-af44-875059e1906d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:27.484114 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f3ea97-a7f2-43d4-af44-875059e1906d None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:27.484393 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f3ea97-a7f2-43d4-af44-875059e1906d None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:27.484693 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f3ea97-a7f2-43d4-af44-875059e1906d None admin] PATH_INFO: `/v3/roles/member` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:27.501923 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f3ea97-a7f2-43d4-af44-875059e1906d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:33:27.510281 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-d2f3ea97-a7f2-43d4-af44-875059e1906d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:33:27.510281 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:33:27.510281 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:33:27.510281 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:33:27.510281 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:33:27.510281 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:33:27.510281 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:33:27.510281 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:33:27.510281 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:33:27.510281 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:33:27.510281 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:33:27.510281 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:33:27.510281 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:33:27.510281 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:33:27.510281 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:33:27.510281 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:33:27.511192 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:33:27.511192 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:33:27.511192 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:33:27.511192 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:33:27.511192 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:33:27.511192 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:33:27.511192 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:33:27.511192 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:33:27.511192 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:33:27.511192 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:33:27.511192 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:33:27.511192 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:33:27.511192 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:33:27.511192 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:33:27.511192 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:33:27.511192 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:33:27.512078 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:33:27.512078 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:33:27.512078 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:33:27.512078 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:33:27.512078 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:33:27.512078 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:33:27.512078 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:33:27.512078 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:33:27.512078 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:33:27.512078 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:33:27.512078 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:33:27.512078 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:33:27.516759 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 59/122] 10.222.0.55 () {56 vars in 1099 bytes} [Wed Aug 24 08:33:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:33:27.520636 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f3ea97-a7f2-43d4-af44-875059e1906d None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:27.599433 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:27.623340 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c809aba2-41f0-4ed1-891f-b69920d570fb None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:27.687639 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5feabd64-ff39-45d9-8fc8-1037b561a3b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:27.688428 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feabd64-ff39-45d9-8fc8-1037b561a3b4 None admin] REQUEST_METHOD: `PUT` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:27.688745 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feabd64-ff39-45d9-8fc8-1037b561a3b4 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:27.689028 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feabd64-ff39-45d9-8fc8-1037b561a3b4 None admin] PATH_INFO: `/v3/projects/f8b7cfbd340b4753bd898ddc0a462a03/users/28800687066c4de989fe1940c9a30412/roles/b870cd1acaa744e1ae1d58b9ac283821` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:27.692446 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5feabd64-ff39-45d9-8fc8-1037b561a3b4 None admin] RBAC: Authorizing `identity:create_grant(project_id=f8b7cfbd340b4753bd898ddc0a462a03, user_id=28800687066c4de989fe1940c9a30412, role_id=b870cd1acaa744e1ae1d58b9ac283821)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:27.694035 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5feabd64-ff39-45d9-8fc8-1037b561a3b4 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:27.748821 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f3ea97-a7f2-43d4-af44-875059e1906d None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:27.790987 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 64/123] 10.222.0.55 () {58 vars in 1439 bytes} [Wed Aug 24 08:33:27 2022] PUT /identity/v3/projects/f8b7cfbd340b4753bd898ddc0a462a03/users/28800687066c4de989fe1940c9a30412/roles/b870cd1acaa744e1ae1d58b9ac283821 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:33:27.791931 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5feabd64-ff39-45d9-8fc8-1037b561a3b4 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:27.803913 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e533f2b6-16f6-4a45-80ae-398091be8076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:27.804573 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e533f2b6-16f6-4a45-80ae-398091be8076 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:27.804875 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e533f2b6-16f6-4a45-80ae-398091be8076 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:27.805152 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e533f2b6-16f6-4a45-80ae-398091be8076 None admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:27.806410 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e533f2b6-16f6-4a45-80ae-398091be8076 None admin] RBAC: Adding query filter params (name=member) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:33:27.806717 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e533f2b6-16f6-4a45-80ae-398091be8076 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:27.808163 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e533f2b6-16f6-4a45-80ae-398091be8076 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:27.824873 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 60/124] 10.222.0.55 () {56 vars in 1101 bytes} [Wed Aug 24 08:33:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:27.824873 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e533f2b6-16f6-4a45-80ae-398091be8076 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:27.905503 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:28.010638 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5feabd64-ff39-45d9-8fc8-1037b561a3b4 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:28.029206 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e533f2b6-16f6-4a45-80ae-398091be8076 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:28.063426 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5f8db1-7a93-40eb-af9a-2623cf56cf3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:28.064460 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5f8db1-7a93-40eb-af9a-2623cf56cf3a None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:28.065001 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5f8db1-7a93-40eb-af9a-2623cf56cf3a None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:28.065494 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5f8db1-7a93-40eb-af9a-2623cf56cf3a None admin] PATH_INFO: `/v3/roles/member` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:28.077972 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5f8db1-7a93-40eb-af9a-2623cf56cf3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:33:28.079726 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78332c0-6903-4f57-88c0-f26a5b9ea938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:28.080373 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78332c0-6903-4f57-88c0-f26a5b9ea938 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:28.080683 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78332c0-6903-4f57-88c0-f26a5b9ea938 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:28.081088 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78332c0-6903-4f57-88c0-f26a5b9ea938 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:28.084448 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-4d5f8db1-7a93-40eb-af9a-2623cf56cf3a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:33:28.084448 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:33:28.084448 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:33:28.084448 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:33:28.084448 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:33:28.084448 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:33:28.084448 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:33:28.084448 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:33:28.084448 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:33:28.084448 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:33:28.084448 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:33:28.084448 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:33:28.084448 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:33:28.084448 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:33:28.084448 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:33:28.084448 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:33:28.085346 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:33:28.085346 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:33:28.085346 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:33:28.085346 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:33:28.085346 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:33:28.085346 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:33:28.085346 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:33:28.085346 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:33:28.085346 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:33:28.085346 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:33:28.085346 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:33:28.085346 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:33:28.085346 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:33:28.085346 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:33:28.085346 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:33:28.085346 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:33:28.086186 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:33:28.086186 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:33:28.086186 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:33:28.086186 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:33:28.086186 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:33:28.086186 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:33:28.086186 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:33:28.086186 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:33:28.086186 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:33:28.086186 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:33:28.086186 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:33:28.086186 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:33:28.089116 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 65/125] 10.222.0.55 () {56 vars in 1099 bytes} [Wed Aug 24 08:33:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:33:28.091705 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5f8db1-7a93-40eb-af9a-2623cf56cf3a None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:28.098907 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:28.099085 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78332c0-6903-4f57-88c0-f26a5b9ea938 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:33:28.107398 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-a78332c0-6903-4f57-88c0-f26a5b9ea938 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 08:33:28.107398 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:33:28.107398 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:33:28.107398 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:33:28.107398 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:33:28.107398 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:33:28.107398 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:33:28.107398 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:33:28.107398 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:33:28.107398 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:33:28.107398 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:33:28.107398 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:33:28.107398 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:33:28.107398 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:33:28.107398 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:33:28.107398 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:33:28.108385 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:33:28.108385 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:33:28.108385 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:33:28.108385 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:33:28.108385 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:33:28.108385 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:33:28.108385 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:33:28.108385 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:33:28.108385 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:33:28.108385 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:33:28.108385 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:33:28.108385 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:33:28.108385 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:33:28.108385 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:33:28.108385 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:33:28.108385 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:33:28.109853 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:33:28.109853 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:33:28.109853 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:33:28.109853 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:33:28.109853 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:33:28.109853 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:33:28.109853 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:33:28.109853 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:33:28.109853 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:33:28.109853 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:33:28.109853 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 08:33:28.109853 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:33:28.112651 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 61/126] 10.222.0.55 () {56 vars in 1114 bytes} [Wed Aug 24 08:33:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:33:28.115253 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78332c0-6903-4f57-88c0-f26a5b9ea938 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:28.271274 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5f8db1-7a93-40eb-af9a-2623cf56cf3a None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:28.293364 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:28.297464 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78332c0-6903-4f57-88c0-f26a5b9ea938 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:28.346923 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87290221-91f6-4911-93bd-f8e57b51b4d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:28.347830 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87290221-91f6-4911-93bd-f8e57b51b4d9 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:28.348250 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87290221-91f6-4911-93bd-f8e57b51b4d9 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:28.348691 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87290221-91f6-4911-93bd-f8e57b51b4d9 None admin] PATH_INFO: `/v3/users/07914530ee0141de9cc889c157da5a4c` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:28.351374 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87290221-91f6-4911-93bd-f8e57b51b4d9 None admin] RBAC: Authorizing `identity:get_user(user_id=07914530ee0141de9cc889c157da5a4c)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:28.353466 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87290221-91f6-4911-93bd-f8e57b51b4d9 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:28.355862 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 66/127] 10.222.0.55 () {56 vars in 1177 bytes} [Wed Aug 24 08:33:28 2022] GET /identity/v3/users/07914530ee0141de9cc889c157da5a4c => generated 275 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:28.362602 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87290221-91f6-4911-93bd-f8e57b51b4d9 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:28.370056 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3796618-d25d-437a-bb21-d33b1395cadf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:28.370700 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3796618-d25d-437a-bb21-d33b1395cadf None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:28.370984 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3796618-d25d-437a-bb21-d33b1395cadf None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:28.371506 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3796618-d25d-437a-bb21-d33b1395cadf None admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:28.372203 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3796618-d25d-437a-bb21-d33b1395cadf None admin] RBAC: Adding query filter params (name=member) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:33:28.372510 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3796618-d25d-437a-bb21-d33b1395cadf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:28.373961 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3796618-d25d-437a-bb21-d33b1395cadf None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:28.390978 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 62/128] 10.222.0.55 () {56 vars in 1101 bytes} [Wed Aug 24 08:33:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:33:28.391918 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3796618-d25d-437a-bb21-d33b1395cadf None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:28.459058 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:28.530939 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87290221-91f6-4911-93bd-f8e57b51b4d9 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:28.536698 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3796618-d25d-437a-bb21-d33b1395cadf None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:28.588500 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10310674-acae-4cb7-bd3e-c1ba584a904e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:28.589451 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10310674-acae-4cb7-bd3e-c1ba584a904e None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:28.589743 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10310674-acae-4cb7-bd3e-c1ba584a904e None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:28.590023 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10310674-acae-4cb7-bd3e-c1ba584a904e None admin] PATH_INFO: `/v3/projects/f8b7cfbd340b4753bd898ddc0a462a03` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:28.591233 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10310674-acae-4cb7-bd3e-c1ba584a904e None admin] RBAC: Authorizing `identity:get_project(project_id=f8b7cfbd340b4753bd898ddc0a462a03)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:28.593308 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10310674-acae-4cb7-bd3e-c1ba584a904e None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:28.595285 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 63/129] 10.222.0.55 () {56 vars in 1186 bytes} [Wed Aug 24 08:33:28 2022] GET /identity/v3/projects/f8b7cfbd340b4753bd898ddc0a462a03 => generated 298 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:28.595899 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10310674-acae-4cb7-bd3e-c1ba584a904e None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:28.629292 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b6908a-4857-4d4a-b193-c21d0910bc90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:28.629938 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b6908a-4857-4d4a-b193-c21d0910bc90 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:28.630226 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b6908a-4857-4d4a-b193-c21d0910bc90 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:28.630503 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b6908a-4857-4d4a-b193-c21d0910bc90 None admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:28.631273 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b6908a-4857-4d4a-b193-c21d0910bc90 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:33:28.631583 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b6908a-4857-4d4a-b193-c21d0910bc90 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:28.633043 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:28.633151 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b6908a-4857-4d4a-b193-c21d0910bc90 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:28.651635 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 67/130] 10.222.0.55 () {56 vars in 1111 bytes} [Wed Aug 24 08:33:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:28.656808 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b6908a-4857-4d4a-b193-c21d0910bc90 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:28.723220 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10310674-acae-4cb7-bd3e-c1ba584a904e None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:28.797200 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e432b8-9c90-439e-8e78-a19192dcd882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:28.797773 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e432b8-9c90-439e-8e78-a19192dcd882 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:28.798179 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e432b8-9c90-439e-8e78-a19192dcd882 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:28.798465 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e432b8-9c90-439e-8e78-a19192dcd882 None admin] PATH_INFO: `/v3/users/07914530ee0141de9cc889c157da5a4c` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:28.802437 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e432b8-9c90-439e-8e78-a19192dcd882 None admin] RBAC: Authorizing `identity:get_user(user_id=07914530ee0141de9cc889c157da5a4c)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:28.804118 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e432b8-9c90-439e-8e78-a19192dcd882 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:28.806383 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 64/131] 10.222.0.55 () {56 vars in 1177 bytes} [Wed Aug 24 08:33:28 2022] GET /identity/v3/users/07914530ee0141de9cc889c157da5a4c => generated 275 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:28.809450 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e432b8-9c90-439e-8e78-a19192dcd882 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:28.845382 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b6908a-4857-4d4a-b193-c21d0910bc90 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:28.885123 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b864af-17a0-4a4b-bee7-2666ec67c551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:28.885812 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b864af-17a0-4a4b-bee7-2666ec67c551 None admin] REQUEST_METHOD: `PUT` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:28.886101 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b864af-17a0-4a4b-bee7-2666ec67c551 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:28.886383 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b864af-17a0-4a4b-bee7-2666ec67c551 None admin] PATH_INFO: `/v3/projects/f8b7cfbd340b4753bd898ddc0a462a03/users/07914530ee0141de9cc889c157da5a4c/roles/b674fd0a592e4e0cada811bba5ceea4d` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:28.890846 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b864af-17a0-4a4b-bee7-2666ec67c551 None admin] RBAC: Authorizing `identity:create_grant(project_id=f8b7cfbd340b4753bd898ddc0a462a03, user_id=07914530ee0141de9cc889c157da5a4c, role_id=b674fd0a592e4e0cada811bba5ceea4d)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:28.894940 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b864af-17a0-4a4b-bee7-2666ec67c551 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:28.902203 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:28.959290 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:28.966034 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e432b8-9c90-439e-8e78-a19192dcd882 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:29.005969 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0dd784-4668-4784-ae6d-0214a692c9af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:29.006644 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0dd784-4668-4784-ae6d-0214a692c9af None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:29.006950 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0dd784-4668-4784-ae6d-0214a692c9af None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:29.007230 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0dd784-4668-4784-ae6d-0214a692c9af None admin] PATH_INFO: `/v3/users/07914530ee0141de9cc889c157da5a4c` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:29.016656 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 68/132] 10.222.0.55 () {58 vars in 1439 bytes} [Wed Aug 24 08:33:28 2022] PUT /identity/v3/projects/f8b7cfbd340b4753bd898ddc0a462a03/users/07914530ee0141de9cc889c157da5a4c/roles/b674fd0a592e4e0cada811bba5ceea4d => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:33:29.017486 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b864af-17a0-4a4b-bee7-2666ec67c551 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:29.022340 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0dd784-4668-4784-ae6d-0214a692c9af None admin] RBAC: Authorizing `identity:get_user(user_id=07914530ee0141de9cc889c157da5a4c)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:29.022340 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0dd784-4668-4784-ae6d-0214a692c9af None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:29.023100 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 65/133] 10.222.0.55 () {56 vars in 1177 bytes} [Wed Aug 24 08:33:28 2022] GET /identity/v3/users/07914530ee0141de9cc889c157da5a4c => generated 275 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:29.028868 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0dd784-4668-4784-ae6d-0214a692c9af None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:29.179748 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0dd784-4668-4784-ae6d-0214a692c9af None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:29.254026 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74f8de1-e321-4ef6-b71e-0319f8c0b140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:29.254956 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74f8de1-e321-4ef6-b71e-0319f8c0b140 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:29.255245 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74f8de1-e321-4ef6-b71e-0319f8c0b140 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:29.255528 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74f8de1-e321-4ef6-b71e-0319f8c0b140 None admin] PATH_INFO: `/v3/projects/f8b7cfbd340b4753bd898ddc0a462a03` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:29.256750 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74f8de1-e321-4ef6-b71e-0319f8c0b140 None admin] RBAC: Authorizing `identity:get_project(project_id=f8b7cfbd340b4753bd898ddc0a462a03)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:29.256851 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b864af-17a0-4a4b-bee7-2666ec67c551 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:29.258208 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74f8de1-e321-4ef6-b71e-0319f8c0b140 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:29.261319 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 66/134] 10.222.0.55 () {56 vars in 1186 bytes} [Wed Aug 24 08:33:29 2022] GET /identity/v3/projects/f8b7cfbd340b4753bd898ddc0a462a03 => generated 298 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:33:29.262588 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30872e92-989c-47cf-8b16-49752421e12e None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:29.262880 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30872e92-989c-47cf-8b16-49752421e12e None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:29.263160 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30872e92-989c-47cf-8b16-49752421e12e None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:29.264376 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 67/135] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:33:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:33:29.268146 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30872e92-989c-47cf-8b16-49752421e12e None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:29.307111 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3fb225-716b-47e2-bd7b-afe8c1d6b1f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:29.308003 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3fb225-716b-47e2-bd7b-afe8c1d6b1f7 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:29.308715 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:29.309313 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3fb225-716b-47e2-bd7b-afe8c1d6b1f7 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:29.309595 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3fb225-716b-47e2-bd7b-afe8c1d6b1f7 None admin] PATH_INFO: `/v3/projects/abc83224889e4e0f8d1675a2cbc7657b` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:29.311060 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3fb225-716b-47e2-bd7b-afe8c1d6b1f7 None admin] RBAC: Authorizing `identity:get_project(project_id=abc83224889e4e0f8d1675a2cbc7657b)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:29.312929 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3fb225-716b-47e2-bd7b-afe8c1d6b1f7 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:29.315672 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 69/136] 10.222.0.55 () {56 vars in 1186 bytes} [Wed Aug 24 08:33:29 2022] GET /identity/v3/projects/abc83224889e4e0f8d1675a2cbc7657b => generated 312 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:29.316966 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7266f1d-574b-456e-beec-46c6a5b6a56c None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:29.325556 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7266f1d-574b-456e-beec-46c6a5b6a56c None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:29.325556 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7266f1d-574b-456e-beec-46c6a5b6a56c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:29.341280 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-f7266f1d-574b-456e-beec-46c6a5b6a56c None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:33:29.342999 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-f7266f1d-574b-456e-beec-46c6a5b6a56c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:29.393279 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30872e92-989c-47cf-8b16-49752421e12e None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:29.428899 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:29.429951 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46bdcaf-a7d0-4067-8eae-a8f169ae5d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:29.431197 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46bdcaf-a7d0-4067-8eae-a8f169ae5d58 None admin] REQUEST_METHOD: `PUT` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:29.431474 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46bdcaf-a7d0-4067-8eae-a8f169ae5d58 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:29.431747 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46bdcaf-a7d0-4067-8eae-a8f169ae5d58 None admin] PATH_INFO: `/v3/projects/f8b7cfbd340b4753bd898ddc0a462a03/users/07914530ee0141de9cc889c157da5a4c/roles/8b5ad96041894155b0138a676ead607e` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:29.434121 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46bdcaf-a7d0-4067-8eae-a8f169ae5d58 None admin] RBAC: Authorizing `identity:create_grant(project_id=f8b7cfbd340b4753bd898ddc0a462a03, user_id=07914530ee0141de9cc889c157da5a4c, role_id=8b5ad96041894155b0138a676ead607e)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:29.435638 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46bdcaf-a7d0-4067-8eae-a8f169ae5d58 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:29.441959 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 70/137] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:33:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:33:29.442893 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7266f1d-574b-456e-beec-46c6a5b6a56c None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:29.527865 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 68/138] 10.222.0.55 () {58 vars in 1439 bytes} [Wed Aug 24 08:33:29 2022] PUT /identity/v3/projects/f8b7cfbd340b4753bd898ddc0a462a03/users/07914530ee0141de9cc889c157da5a4c/roles/8b5ad96041894155b0138a676ead607e => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:33:29.559765 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7266f1d-574b-456e-beec-46c6a5b6a56c None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:29.585727 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915acfa9-48da-40c6-b6be-13bc2bbd8e4d None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:29.586058 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915acfa9-48da-40c6-b6be-13bc2bbd8e4d None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:29.586280 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915acfa9-48da-40c6-b6be-13bc2bbd8e4d None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:29.588247 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 69/139] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:33:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:33:29.603538 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16470e23-1389-4725-b261-c0ec089f565a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:29.604239 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16470e23-1389-4725-b261-c0ec089f565a None admin] REQUEST_METHOD: `PUT` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:29.604525 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16470e23-1389-4725-b261-c0ec089f565a None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:29.604823 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16470e23-1389-4725-b261-c0ec089f565a None admin] PATH_INFO: `/v3/projects/abc83224889e4e0f8d1675a2cbc7657b/users/07914530ee0141de9cc889c157da5a4c/roles/b674fd0a592e4e0cada811bba5ceea4d` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:29.606810 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16470e23-1389-4725-b261-c0ec089f565a None admin] RBAC: Authorizing `identity:create_grant(project_id=abc83224889e4e0f8d1675a2cbc7657b, user_id=07914530ee0141de9cc889c157da5a4c, role_id=b674fd0a592e4e0cada811bba5ceea4d)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:29.607886 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98e9700-99d3-4361-95fb-c5f363633408 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:29.608174 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98e9700-99d3-4361-95fb-c5f363633408 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:29.608367 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16470e23-1389-4725-b261-c0ec089f565a None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:29.608524 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98e9700-99d3-4361-95fb-c5f363633408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:29.647706 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-f98e9700-99d3-4361-95fb-c5f363633408 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:33:29.649154 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-f98e9700-99d3-4361-95fb-c5f363633408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:29.697421 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 71/140] 10.222.0.55 () {58 vars in 1439 bytes} [Wed Aug 24 08:33:29 2022] PUT /identity/v3/projects/abc83224889e4e0f8d1675a2cbc7657b/users/07914530ee0141de9cc889c157da5a4c/roles/b674fd0a592e4e0cada811bba5ceea4d => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:33:29.765966 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 70/141] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:33:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:33:29.781281 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16470e23-1389-4725-b261-c0ec089f565a None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:29.849625 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:29.904711 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16470e23-1389-4725-b261-c0ec089f565a None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:29.944760 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa55aa7-afba-4d8d-84d7-ee3c79545eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:29.945493 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa55aa7-afba-4d8d-84d7-ee3c79545eae None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:29.945777 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa55aa7-afba-4d8d-84d7-ee3c79545eae None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:29.946086 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa55aa7-afba-4d8d-84d7-ee3c79545eae None admin] PATH_INFO: `/v3/domains/default` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:29.947270 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa55aa7-afba-4d8d-84d7-ee3c79545eae None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:29.948887 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa55aa7-afba-4d8d-84d7-ee3c79545eae None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:29.950521 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 72/142] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:33:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:29.956612 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98e9700-99d3-4361-95fb-c5f363633408 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:29.987658 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:30.077724 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98e9700-99d3-4361-95fb-c5f363633408 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:30.119403 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f039a17-27dc-4f78-9dbd-7fb8c74f2ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:30.120030 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f039a17-27dc-4f78-9dbd-7fb8c74f2ddb None admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:30.120317 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f039a17-27dc-4f78-9dbd-7fb8c74f2ddb None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:30.120668 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f039a17-27dc-4f78-9dbd-7fb8c74f2ddb None admin] PATH_INFO: `/v3/users` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:30.121336 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f039a17-27dc-4f78-9dbd-7fb8c74f2ddb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:30.123047 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f039a17-27dc-4f78-9dbd-7fb8c74f2ddb None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:30.177715 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa55aa7-afba-4d8d-84d7-ee3c79545eae None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:30.258774 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 71/143] 10.222.0.55 () {60 vars in 1132 bytes} [Wed Aug 24 08:33:29 2022] POST /identity/v3/users => generated 283 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:33:30.340154 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa55aa7-afba-4d8d-84d7-ee3c79545eae None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:30.351954 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fce176-18b0-4fcb-862e-38377489cf1a None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:30.352292 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fce176-18b0-4fcb-862e-38377489cf1a None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:30.354076 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fce176-18b0-4fcb-862e-38377489cf1a None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:30.355175 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 72/144] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:33:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:33:30.367563 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee53a2cb-7e9a-4528-a331-43e2701037dc None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:30.367915 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee53a2cb-7e9a-4528-a331-43e2701037dc None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:30.368159 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee53a2cb-7e9a-4528-a331-43e2701037dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:30.370872 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:30.408712 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-ee53a2cb-7e9a-4528-a331-43e2701037dc None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:33:30.410218 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-ee53a2cb-7e9a-4528-a331-43e2701037dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:30.418871 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4f319a-ed4a-4c5c-8621-b8d520c3f0f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:30.419887 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4f319a-ed4a-4c5c-8621-b8d520c3f0f0 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:30.420386 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4f319a-ed4a-4c5c-8621-b8d520c3f0f0 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:30.420883 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4f319a-ed4a-4c5c-8621-b8d520c3f0f0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:30.452284 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4f319a-ed4a-4c5c-8621-b8d520c3f0f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:33:30.457557 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-9c4f319a-ed4a-4c5c-8621-b8d520c3f0f0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 08:33:30.457557 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:33:30.457557 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:33:30.457557 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:33:30.457557 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:33:30.457557 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:33:30.457557 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:33:30.457557 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:33:30.457557 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:33:30.457557 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:33:30.457557 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:33:30.457557 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:33:30.457557 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:33:30.457557 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:33:30.457557 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:33:30.457557 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:33:30.458468 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:33:30.458468 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:33:30.458468 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:33:30.458468 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:33:30.458468 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:33:30.458468 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:33:30.458468 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:33:30.458468 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:33:30.458468 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:33:30.458468 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:33:30.458468 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:33:30.458468 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:33:30.458468 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:33:30.458468 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:33:30.458468 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:33:30.458468 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:33:30.459208 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:33:30.459208 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:33:30.459208 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:33:30.459208 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:33:30.459208 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:33:30.459208 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:33:30.459208 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:33:30.459208 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:33:30.459208 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:33:30.459208 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:33:30.459208 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 08:33:30.459208 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:33:30.468290 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 73/145] 10.222.0.55 () {56 vars in 1099 bytes} [Wed Aug 24 08:33:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:33:30.474541 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4f319a-ed4a-4c5c-8621-b8d520c3f0f0 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:30.510155 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 73/146] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:33:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 08:33:30.647441 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4f319a-ed4a-4c5c-8621-b8d520c3f0f0 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:30.689099 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5267cf8b-08de-4dbe-a3f8-43780d2b6555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:30.689831 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5267cf8b-08de-4dbe-a3f8-43780d2b6555 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:30.690186 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5267cf8b-08de-4dbe-a3f8-43780d2b6555 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:30.690531 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5267cf8b-08de-4dbe-a3f8-43780d2b6555 None admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:30.691331 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5267cf8b-08de-4dbe-a3f8-43780d2b6555 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:33:30.691699 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5267cf8b-08de-4dbe-a3f8-43780d2b6555 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:30.693479 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5267cf8b-08de-4dbe-a3f8-43780d2b6555 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:30.705113 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 74/147] 10.222.0.55 () {56 vars in 1101 bytes} [Wed Aug 24 08:33:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:33:30.711075 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee53a2cb-7e9a-4528-a331-43e2701037dc None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:30.734809 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:30.851967 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5267cf8b-08de-4dbe-a3f8-43780d2b6555 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:30.879566 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee53a2cb-7e9a-4528-a331-43e2701037dc None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:30.920999 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:30.927702 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3bd09d-ead2-41bc-aee7-520e9729f8ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:30.928675 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3bd09d-ead2-41bc-aee7-520e9729f8ee None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:30.929140 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3bd09d-ead2-41bc-aee7-520e9729f8ee None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:30.929582 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3bd09d-ead2-41bc-aee7-520e9729f8ee None admin] PATH_INFO: `/v3/users/5b29b7c0626f4c3da644d63cf7510055` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:30.938255 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3bd09d-ead2-41bc-aee7-520e9729f8ee None admin] RBAC: Authorizing `identity:get_user(user_id=5b29b7c0626f4c3da644d63cf7510055)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:30.938255 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3bd09d-ead2-41bc-aee7-520e9729f8ee None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:30.938255 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 74/148] 10.222.0.55 () {56 vars in 1177 bytes} [Wed Aug 24 08:33:30 2022] GET /identity/v3/users/5b29b7c0626f4c3da644d63cf7510055 => generated 289 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:30.941091 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3bd09d-ead2-41bc-aee7-520e9729f8ee None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:30.999274 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5267cf8b-08de-4dbe-a3f8-43780d2b6555 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:31.048760 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:31.059342 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ede8bb7-e27a-4b2f-8aa5-28c483ef1982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:31.062119 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ede8bb7-e27a-4b2f-8aa5-28c483ef1982 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:31.062958 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ede8bb7-e27a-4b2f-8aa5-28c483ef1982 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:31.064383 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ede8bb7-e27a-4b2f-8aa5-28c483ef1982 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:31.082631 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ede8bb7-e27a-4b2f-8aa5-28c483ef1982 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:33:31.089569 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-3ede8bb7-e27a-4b2f-8aa5-28c483ef1982 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:33:31.089569 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:33:31.089569 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:33:31.089569 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:33:31.089569 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:33:31.089569 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:33:31.089569 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:33:31.089569 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:33:31.089569 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:33:31.089569 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:33:31.089569 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:33:31.089569 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:33:31.089569 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:33:31.089569 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:33:31.089569 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:33:31.089569 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:33:31.090471 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:33:31.090471 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:33:31.090471 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:33:31.090471 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:33:31.090471 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:33:31.090471 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:33:31.090471 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:33:31.090471 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:33:31.090471 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:33:31.090471 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:33:31.090471 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:33:31.090471 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:33:31.090471 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:33:31.090471 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:33:31.090471 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:33:31.090471 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:33:31.091313 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:33:31.091313 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:33:31.091313 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:33:31.091313 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:33:31.091313 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:33:31.091313 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:33:31.091313 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:33:31.091313 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:33:31.091313 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:33:31.091313 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:33:31.091313 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:33:31.091313 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:33:31.093889 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 75/149] 10.222.0.55 () {56 vars in 1096 bytes} [Wed Aug 24 08:33:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:33:31.100480 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ede8bb7-e27a-4b2f-8aa5-28c483ef1982 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:31.107624 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3bd09d-ead2-41bc-aee7-520e9729f8ee None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:31.171713 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ac42b7-8ca0-4040-a8fe-76444547ef30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:31.172784 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ac42b7-8ca0-4040-a8fe-76444547ef30 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:31.173264 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ac42b7-8ca0-4040-a8fe-76444547ef30 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:31.173715 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ac42b7-8ca0-4040-a8fe-76444547ef30 None admin] PATH_INFO: `/v3/projects/f8b7cfbd340b4753bd898ddc0a462a03` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:31.175246 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ac42b7-8ca0-4040-a8fe-76444547ef30 None admin] RBAC: Authorizing `identity:get_project(project_id=f8b7cfbd340b4753bd898ddc0a462a03)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:31.177156 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ac42b7-8ca0-4040-a8fe-76444547ef30 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:31.179274 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 75/150] 10.222.0.55 () {56 vars in 1186 bytes} [Wed Aug 24 08:33:30 2022] GET /identity/v3/projects/f8b7cfbd340b4753bd898ddc0a462a03 => generated 298 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:33:31.186266 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ac42b7-8ca0-4040-a8fe-76444547ef30 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:31.274652 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ede8bb7-e27a-4b2f-8aa5-28c483ef1982 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:31.276614 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:31.349529 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7501d0-b7f3-485f-bae9-ea598a5fff60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:31.350198 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7501d0-b7f3-485f-bae9-ea598a5fff60 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:31.350467 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7501d0-b7f3-485f-bae9-ea598a5fff60 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:31.350745 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7501d0-b7f3-485f-bae9-ea598a5fff60 None admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:31.351516 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7501d0-b7f3-485f-bae9-ea598a5fff60 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:33:31.351817 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7501d0-b7f3-485f-bae9-ea598a5fff60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:31.353801 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7501d0-b7f3-485f-bae9-ea598a5fff60 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:31.356927 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ac42b7-8ca0-4040-a8fe-76444547ef30 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:31.375266 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 76/151] 10.222.0.55 () {56 vars in 1099 bytes} [Wed Aug 24 08:33:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:33:31.381715 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7501d0-b7f3-485f-bae9-ea598a5fff60 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:31.435328 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d0f7de-e469-40cb-8c74-a26842deaf10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:31.436101 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d0f7de-e469-40cb-8c74-a26842deaf10 None admin] REQUEST_METHOD: `PUT` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:31.436384 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d0f7de-e469-40cb-8c74-a26842deaf10 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:31.436684 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d0f7de-e469-40cb-8c74-a26842deaf10 None admin] PATH_INFO: `/v3/projects/f8b7cfbd340b4753bd898ddc0a462a03/users/5b29b7c0626f4c3da644d63cf7510055/roles/e308031ba9a84b79bcdf501455ebe0d7` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:31.439587 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d0f7de-e469-40cb-8c74-a26842deaf10 None admin] RBAC: Authorizing `identity:create_grant(project_id=f8b7cfbd340b4753bd898ddc0a462a03, user_id=5b29b7c0626f4c3da644d63cf7510055, role_id=e308031ba9a84b79bcdf501455ebe0d7)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:31.441309 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d0f7de-e469-40cb-8c74-a26842deaf10 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:31.531995 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 76/152] 10.222.0.55 () {58 vars in 1439 bytes} [Wed Aug 24 08:33:31 2022] PUT /identity/v3/projects/f8b7cfbd340b4753bd898ddc0a462a03/users/5b29b7c0626f4c3da644d63cf7510055/roles/e308031ba9a84b79bcdf501455ebe0d7 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:33:31.554474 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7501d0-b7f3-485f-bae9-ea598a5fff60 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:31.609735 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9cf1e5-7819-48ee-be8d-fc17f0a36eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:31.610455 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9cf1e5-7819-48ee-be8d-fc17f0a36eee None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:31.610740 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9cf1e5-7819-48ee-be8d-fc17f0a36eee None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:31.611015 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9cf1e5-7819-48ee-be8d-fc17f0a36eee None admin] PATH_INFO: `/v3/users/28800687066c4de989fe1940c9a30412` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:31.612306 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9cf1e5-7819-48ee-be8d-fc17f0a36eee None admin] RBAC: Authorizing `identity:get_user(user_id=28800687066c4de989fe1940c9a30412)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:31.613894 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9cf1e5-7819-48ee-be8d-fc17f0a36eee None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:31.615853 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 77/153] 10.222.0.55 () {56 vars in 1177 bytes} [Wed Aug 24 08:33:31 2022] GET /identity/v3/users/28800687066c4de989fe1940c9a30412 => generated 247 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:31.622046 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d0f7de-e469-40cb-8c74-a26842deaf10 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:31.626302 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:31.744689 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d0f7de-e469-40cb-8c74-a26842deaf10 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:31.794443 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b067636d-6bf8-4951-98b0-0cc8f3053132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:31.795442 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b067636d-6bf8-4951-98b0-0cc8f3053132 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:31.795926 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b067636d-6bf8-4951-98b0-0cc8f3053132 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:31.796399 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b067636d-6bf8-4951-98b0-0cc8f3053132 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:31.872298 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b067636d-6bf8-4951-98b0-0cc8f3053132 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:31.874230 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b067636d-6bf8-4951-98b0-0cc8f3053132 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:31.878033 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:31.989425 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 77/154] 10.222.0.55 () {58 vars in 1282 bytes} [Wed Aug 24 08:33:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:33:31.995307 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9cf1e5-7819-48ee-be8d-fc17f0a36eee None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:32.032912 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:32.112480 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9cf1e5-7819-48ee-be8d-fc17f0a36eee None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:32.155680 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf48917-385f-4dd8-ab2e-fa232e36b83b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:32.156239 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf48917-385f-4dd8-ab2e-fa232e36b83b None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:32.156522 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf48917-385f-4dd8-ab2e-fa232e36b83b None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:32.156814 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf48917-385f-4dd8-ab2e-fa232e36b83b None admin] PATH_INFO: `/v3/projects/f8b7cfbd340b4753bd898ddc0a462a03` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:32.157973 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf48917-385f-4dd8-ab2e-fa232e36b83b None admin] RBAC: Authorizing `identity:get_project(project_id=f8b7cfbd340b4753bd898ddc0a462a03)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:32.159412 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf48917-385f-4dd8-ab2e-fa232e36b83b None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:32.161377 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 78/155] 10.222.0.55 () {56 vars in 1186 bytes} [Wed Aug 24 08:33:31 2022] GET /identity/v3/projects/f8b7cfbd340b4753bd898ddc0a462a03 => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:32.167347 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b067636d-6bf8-4951-98b0-0cc8f3053132 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:32.278383 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b067636d-6bf8-4951-98b0-0cc8f3053132 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:32.280869 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:32.320330 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7d7bb7-1c67-4021-b511-ac1d2a3e7b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:32.320990 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7d7bb7-1c67-4021-b511-ac1d2a3e7b3a None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:32.322513 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7d7bb7-1c67-4021-b511-ac1d2a3e7b3a None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:32.322789 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7d7bb7-1c67-4021-b511-ac1d2a3e7b3a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:32.323531 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7d7bb7-1c67-4021-b511-ac1d2a3e7b3a None admin] RBAC: Adding query filter params (role.id=b870cd1acaa744e1ae1d58b9ac283821, scope.project.id=f8b7cfbd340b4753bd898ddc0a462a03, user.id=28800687066c4de989fe1940c9a30412) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:33:32.323840 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7d7bb7-1c67-4021-b511-ac1d2a3e7b3a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:32.325338 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7d7bb7-1c67-4021-b511-ac1d2a3e7b3a None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:32.332253 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 78/156] 10.222.0.55 () {56 vars in 1374 bytes} [Wed Aug 24 08:33:32 2022] GET /identity/v3/role_assignments?user.id=28800687066c4de989fe1940c9a30412&scope.project.id=f8b7cfbd340b4753bd898ddc0a462a03&role.id=b870cd1acaa744e1ae1d58b9ac283821 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:32.539514 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f9cacb-6f33-4790-9486-6a05160b95f0 None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:32.539701 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f9cacb-6f33-4790-9486-6a05160b95f0 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:32.540096 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f9cacb-6f33-4790-9486-6a05160b95f0 None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:32.541508 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 79/157] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:33:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:33:32.547422 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa31494c-11f4-4bef-b579-44088d0c583a None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:32.547715 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa31494c-11f4-4bef-b579-44088d0c583a None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:32.547980 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa31494c-11f4-4bef-b579-44088d0c583a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:32.570630 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-aa31494c-11f4-4bef-b579-44088d0c583a None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:33:32.572402 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-aa31494c-11f4-4bef-b579-44088d0c583a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:32.649188 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 79/158] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:33:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:33:33.248719 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f9cacb-6f33-4790-9486-6a05160b95f0 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:33.278369 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744d83f8-2d5e-4682-b819-d0e99ca0cf36 None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:33.278865 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744d83f8-2d5e-4682-b819-d0e99ca0cf36 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:33.278938 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744d83f8-2d5e-4682-b819-d0e99ca0cf36 None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:33.280424 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 80/159] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:33:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:33:33.291019 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed0cd6f-4f3a-46dd-a54a-1eb662c7020e None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:33.291314 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed0cd6f-4f3a-46dd-a54a-1eb662c7020e None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:33.291564 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed0cd6f-4f3a-46dd-a54a-1eb662c7020e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:33.346197 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-bed0cd6f-4f3a-46dd-a54a-1eb662c7020e None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:33:33.347415 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-bed0cd6f-4f3a-46dd-a54a-1eb662c7020e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:33.459145 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:33.469339 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f9cacb-6f33-4790-9486-6a05160b95f0 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:33.537650 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75194648-a15b-4b0b-8a18-b044086c6dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:33.538313 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75194648-a15b-4b0b-8a18-b044086c6dba None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:33.538701 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75194648-a15b-4b0b-8a18-b044086c6dba None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:33.539053 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75194648-a15b-4b0b-8a18-b044086c6dba None admin] PATH_INFO: `/v3/roles/member` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:33.553859 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75194648-a15b-4b0b-8a18-b044086c6dba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:33:33.561347 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-75194648-a15b-4b0b-8a18-b044086c6dba None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:33:33.561347 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:33:33.561347 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:33:33.561347 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:33:33.561347 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:33:33.561347 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:33:33.561347 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:33:33.561347 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:33:33.561347 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:33:33.561347 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:33:33.561347 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:33:33.561347 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:33:33.561347 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:33:33.561347 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:33:33.561347 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:33:33.561347 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:33:33.562302 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:33:33.562302 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:33:33.562302 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:33:33.562302 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:33:33.562302 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:33:33.562302 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:33:33.562302 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:33:33.562302 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:33:33.562302 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:33:33.562302 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:33:33.562302 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:33:33.562302 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:33:33.562302 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:33:33.562302 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:33:33.562302 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:33:33.562302 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:33:33.563191 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:33:33.563191 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:33:33.563191 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:33:33.563191 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:33:33.563191 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:33:33.563191 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:33:33.563191 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:33:33.563191 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:33:33.563191 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:33:33.563191 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:33:33.563191 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:33:33.563191 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:33:33.563191 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 80/160] 10.222.0.55 () {56 vars in 1099 bytes} [Wed Aug 24 08:33:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:33:33.564887 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058de72a-2e62-41d4-83c7-1c59a0ba0a38 None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:33.565009 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058de72a-2e62-41d4-83c7-1c59a0ba0a38 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:33.565214 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058de72a-2e62-41d4-83c7-1c59a0ba0a38 None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:33.567051 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 81/161] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:33:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:33:33.578244 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fa484e-c898-4ba4-84ed-71f98e4079f4 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:33.578531 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fa484e-c898-4ba4-84ed-71f98e4079f4 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:33.578801 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fa484e-c898-4ba4-84ed-71f98e4079f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:33.596067 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 81/162] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:33:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 08:33:33.600574 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed0cd6f-4f3a-46dd-a54a-1eb662c7020e None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:33.616603 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-b2fa484e-c898-4ba4-84ed-71f98e4079f4 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:33:33.616603 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-b2fa484e-c898-4ba4-84ed-71f98e4079f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:33.713692 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:33.754152 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 82/163] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:33:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:33:33.839898 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed0cd6f-4f3a-46dd-a54a-1eb662c7020e None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:33.896590 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cf7fff-d4dd-4ae5-9dd1-b08fb2d90c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:33.897232 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cf7fff-d4dd-4ae5-9dd1-b08fb2d90c32 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:33.897515 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cf7fff-d4dd-4ae5-9dd1-b08fb2d90c32 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:33.897786 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cf7fff-d4dd-4ae5-9dd1-b08fb2d90c32 None admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:33.898456 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cf7fff-d4dd-4ae5-9dd1-b08fb2d90c32 None admin] RBAC: Adding query filter params (name=member) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:33:33.898750 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cf7fff-d4dd-4ae5-9dd1-b08fb2d90c32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:33.900713 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cf7fff-d4dd-4ae5-9dd1-b08fb2d90c32 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:33.914044 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 82/164] 10.222.0.55 () {56 vars in 1101 bytes} [Wed Aug 24 08:33:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:33.921256 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fa484e-c898-4ba4-84ed-71f98e4079f4 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:34.030727 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cf7fff-d4dd-4ae5-9dd1-b08fb2d90c32 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:34.059743 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:34.070425 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fa484e-c898-4ba4-84ed-71f98e4079f4 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:34.161305 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41346697-e85a-4b1f-859f-ae4afb08bad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:34.161967 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41346697-e85a-4b1f-859f-ae4afb08bad4 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:34.162254 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41346697-e85a-4b1f-859f-ae4afb08bad4 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:34.162533 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41346697-e85a-4b1f-859f-ae4afb08bad4 None admin] PATH_INFO: `/v3/users/07914530ee0141de9cc889c157da5a4c` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:34.163736 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41346697-e85a-4b1f-859f-ae4afb08bad4 None admin] RBAC: Authorizing `identity:get_user(user_id=07914530ee0141de9cc889c157da5a4c)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:34.165270 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41346697-e85a-4b1f-859f-ae4afb08bad4 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:34.166874 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 83/165] 10.222.0.55 () {56 vars in 1177 bytes} [Wed Aug 24 08:33:33 2022] GET /identity/v3/users/07914530ee0141de9cc889c157da5a4c => generated 275 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:34.172567 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41346697-e85a-4b1f-859f-ae4afb08bad4 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:34.231353 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:34.250513 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cf7fff-d4dd-4ae5-9dd1-b08fb2d90c32 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:34.318506 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cc23c3-cb18-44c0-b62f-5514553414a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:34.319139 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cc23c3-cb18-44c0-b62f-5514553414a0 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:34.319414 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cc23c3-cb18-44c0-b62f-5514553414a0 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:34.319688 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cc23c3-cb18-44c0-b62f-5514553414a0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:34.343256 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cc23c3-cb18-44c0-b62f-5514553414a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:33:34.349191 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-e9cc23c3-cb18-44c0-b62f-5514553414a0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 08:33:34.349191 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:33:34.349191 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:33:34.349191 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:33:34.349191 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:33:34.349191 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:33:34.349191 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:33:34.349191 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:33:34.349191 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:33:34.349191 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:33:34.349191 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:33:34.349191 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:33:34.349191 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:33:34.349191 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:33:34.349191 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:33:34.349191 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:33:34.350165 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:33:34.350165 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:33:34.350165 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:33:34.350165 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:33:34.350165 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:33:34.350165 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:33:34.350165 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:33:34.350165 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:33:34.350165 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:33:34.350165 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:33:34.350165 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:33:34.350165 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:33:34.350165 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:33:34.350165 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:33:34.350165 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:33:34.350165 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:33:34.350952 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:33:34.350952 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:33:34.350952 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:33:34.350952 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:33:34.350952 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:33:34.350952 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:33:34.350952 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:33:34.350952 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:33:34.350952 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:33:34.350952 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:33:34.350952 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 08:33:34.350952 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:33:34.353602 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 83/166] 10.222.0.55 () {56 vars in 1114 bytes} [Wed Aug 24 08:33:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:33:34.357494 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cc23c3-cb18-44c0-b62f-5514553414a0 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:34.386440 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41346697-e85a-4b1f-859f-ae4afb08bad4 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:34.449128 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9950508f-d79b-4a7e-a424-098dbb3e057f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:34.449743 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9950508f-d79b-4a7e-a424-098dbb3e057f None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:34.450022 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9950508f-d79b-4a7e-a424-098dbb3e057f None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:34.450293 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9950508f-d79b-4a7e-a424-098dbb3e057f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:34.509475 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:34.540123 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cc23c3-cb18-44c0-b62f-5514553414a0 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:34.586448 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539ee36f-6576-4662-9982-799a8712429a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:34.587059 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539ee36f-6576-4662-9982-799a8712429a None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:34.587344 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539ee36f-6576-4662-9982-799a8712429a None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:34.587624 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539ee36f-6576-4662-9982-799a8712429a None admin] PATH_INFO: `/v3/roles/member` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:34.599158 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9950508f-d79b-4a7e-a424-098dbb3e057f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:34.600676 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9950508f-d79b-4a7e-a424-098dbb3e057f None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:34.615926 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539ee36f-6576-4662-9982-799a8712429a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:33:34.621316 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-539ee36f-6576-4662-9982-799a8712429a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:33:34.621316 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:33:34.621316 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:33:34.621316 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:33:34.621316 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:33:34.621316 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:33:34.621316 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:33:34.621316 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:33:34.621316 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:33:34.621316 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:33:34.621316 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:33:34.621316 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:33:34.621316 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:33:34.621316 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:33:34.621316 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:33:34.621316 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:33:34.622479 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:33:34.622479 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:33:34.622479 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:33:34.622479 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:33:34.622479 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:33:34.622479 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:33:34.622479 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:33:34.622479 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:33:34.622479 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:33:34.622479 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:33:34.622479 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:33:34.622479 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:33:34.622479 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:33:34.622479 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:33:34.622479 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:33:34.622479 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:33:34.623257 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:33:34.623257 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:33:34.623257 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:33:34.623257 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:33:34.623257 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:33:34.623257 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:33:34.623257 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:33:34.623257 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:33:34.623257 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:33:34.623257 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:33:34.623257 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:33:34.623257 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:33:34.623257 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 84/167] 10.222.0.55 () {56 vars in 1099 bytes} [Wed Aug 24 08:33:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:33:34.633474 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539ee36f-6576-4662-9982-799a8712429a None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:34.656253 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:34.728379 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 84/168] 10.222.0.55 () {58 vars in 1282 bytes} [Wed Aug 24 08:33:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 556 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 08:33:34.729780 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45542af-186f-4d83-a444-37392687b574 None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:34.730131 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45542af-186f-4d83-a444-37392687b574 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:34.730409 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45542af-186f-4d83-a444-37392687b574 None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:34.731962 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 85/169] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:33:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:33:34.732805 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45542af-186f-4d83-a444-37392687b574 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:34.799554 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:34.831534 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539ee36f-6576-4662-9982-799a8712429a None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:34.848132 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45542af-186f-4d83-a444-37392687b574 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:34.894038 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15b4b52-2612-461d-8b05-d748ecdbfdea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:34.895488 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f67fd8-563f-4266-9388-65dffe359e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:34.896123 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f67fd8-563f-4266-9388-65dffe359e53 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:34.896414 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f67fd8-563f-4266-9388-65dffe359e53 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:34.896721 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f67fd8-563f-4266-9388-65dffe359e53 None admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:34.897406 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f67fd8-563f-4266-9388-65dffe359e53 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:33:34.897712 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f67fd8-563f-4266-9388-65dffe359e53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:34.899123 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f67fd8-563f-4266-9388-65dffe359e53 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:34.899526 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b4b52-2612-461d-8b05-d748ecdbfdea None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:34.899813 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b4b52-2612-461d-8b05-d748ecdbfdea None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:34.900079 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b4b52-2612-461d-8b05-d748ecdbfdea None admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:34.900861 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15b4b52-2612-461d-8b05-d748ecdbfdea None admin] RBAC: Adding query filter params (name=member) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:33:34.901165 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15b4b52-2612-461d-8b05-d748ecdbfdea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:34.902617 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15b4b52-2612-461d-8b05-d748ecdbfdea None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:34.917980 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 86/170] 10.222.0.55 () {56 vars in 1101 bytes} [Wed Aug 24 08:33:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:34.919835 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15b4b52-2612-461d-8b05-d748ecdbfdea None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:34.922862 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 85/171] 10.222.0.55 () {56 vars in 1111 bytes} [Wed Aug 24 08:33:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:34.925474 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390546fe-2910-4086-be48-25a8d59a54e7 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:34.925835 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390546fe-2910-4086-be48-25a8d59a54e7 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:34.926114 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390546fe-2910-4086-be48-25a8d59a54e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:34.955390 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-390546fe-2910-4086-be48-25a8d59a54e7 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:33:34.957170 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-390546fe-2910-4086-be48-25a8d59a54e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:35.048941 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15b4b52-2612-461d-8b05-d748ecdbfdea None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:35.070266 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 86/172] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:33:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:33:35.071125 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390546fe-2910-4086-be48-25a8d59a54e7 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:35.073622 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:35.074537 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:35.090644 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde81222-70a5-4a6e-a140-aae9bf8b3627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:35.091289 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde81222-70a5-4a6e-a140-aae9bf8b3627 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:35.092912 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde81222-70a5-4a6e-a140-aae9bf8b3627 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:35.093450 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde81222-70a5-4a6e-a140-aae9bf8b3627 None admin] PATH_INFO: `/v3/projects/f8b7cfbd340b4753bd898ddc0a462a03` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:35.096072 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde81222-70a5-4a6e-a140-aae9bf8b3627 None admin] RBAC: Authorizing `identity:get_project(project_id=f8b7cfbd340b4753bd898ddc0a462a03)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:35.099556 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde81222-70a5-4a6e-a140-aae9bf8b3627 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:35.102514 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 87/173] 10.222.0.55 () {56 vars in 1186 bytes} [Wed Aug 24 08:33:34 2022] GET /identity/v3/projects/f8b7cfbd340b4753bd898ddc0a462a03 => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:35.104933 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde81222-70a5-4a6e-a140-aae9bf8b3627 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:35.261222 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390546fe-2910-4086-be48-25a8d59a54e7 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:35.276958 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde81222-70a5-4a6e-a140-aae9bf8b3627 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:35.297617 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780b8b44-6ebb-48b8-ac64-db528305fcfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:35.298261 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b8b44-6ebb-48b8-ac64-db528305fcfb None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:35.298565 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b8b44-6ebb-48b8-ac64-db528305fcfb None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:35.298909 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b8b44-6ebb-48b8-ac64-db528305fcfb None admin] PATH_INFO: `/v3/users/07914530ee0141de9cc889c157da5a4c` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:35.300091 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780b8b44-6ebb-48b8-ac64-db528305fcfb None admin] RBAC: Authorizing `identity:get_user(user_id=07914530ee0141de9cc889c157da5a4c)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:35.301719 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780b8b44-6ebb-48b8-ac64-db528305fcfb None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:35.303606 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 87/174] 10.222.0.55 () {56 vars in 1177 bytes} [Wed Aug 24 08:33:35 2022] GET /identity/v3/users/07914530ee0141de9cc889c157da5a4c => generated 275 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:35.304998 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10328ef-7235-49ea-aae6-db809368da73 None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:35.305307 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10328ef-7235-49ea-aae6-db809368da73 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:35.305557 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10328ef-7235-49ea-aae6-db809368da73 None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:35.307355 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 88/175] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:33:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:33:35.308201 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10328ef-7235-49ea-aae6-db809368da73 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:35.315954 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b675ca90-e4ad-49ab-95d9-8c34d7ea3059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:35.316651 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b675ca90-e4ad-49ab-95d9-8c34d7ea3059 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:35.316938 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b675ca90-e4ad-49ab-95d9-8c34d7ea3059 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:35.317215 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b675ca90-e4ad-49ab-95d9-8c34d7ea3059 None admin] PATH_INFO: `/v3/users/07914530ee0141de9cc889c157da5a4c` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:35.319227 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b675ca90-e4ad-49ab-95d9-8c34d7ea3059 None admin] RBAC: Authorizing `identity:get_user(user_id=07914530ee0141de9cc889c157da5a4c)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:35.320773 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b675ca90-e4ad-49ab-95d9-8c34d7ea3059 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:35.322525 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 88/176] 10.222.0.55 () {56 vars in 1177 bytes} [Wed Aug 24 08:33:35 2022] GET /identity/v3/users/07914530ee0141de9cc889c157da5a4c => generated 275 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:35.323847 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d7e902-d103-4221-bafc-ed40f9687dd3 None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:35.324046 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d7e902-d103-4221-bafc-ed40f9687dd3 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:35.324324 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d7e902-d103-4221-bafc-ed40f9687dd3 None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:35.327659 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 89/177] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:33:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:33:35.327659 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ac15e9-5f6c-4fd2-a34d-ed3f42f1281e None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:35.327659 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ac15e9-5f6c-4fd2-a34d-ed3f42f1281e None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:35.327659 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ac15e9-5f6c-4fd2-a34d-ed3f42f1281e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:35.346920 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-87ac15e9-5f6c-4fd2-a34d-ed3f42f1281e None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:33:35.348134 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-87ac15e9-5f6c-4fd2-a34d-ed3f42f1281e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:35.391994 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:35.403377 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:35.419600 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 90/178] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:33:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:33:35.420453 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ac15e9-5f6c-4fd2-a34d-ed3f42f1281e None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:35.425046 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10328ef-7235-49ea-aae6-db809368da73 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:35.461924 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b53d3d-0171-45fc-9f78-936cf9deff01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:35.463061 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b53d3d-0171-45fc-9f78-936cf9deff01 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:35.463341 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b53d3d-0171-45fc-9f78-936cf9deff01 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:35.463985 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b53d3d-0171-45fc-9f78-936cf9deff01 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:35.464761 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b53d3d-0171-45fc-9f78-936cf9deff01 None admin] RBAC: Adding query filter params (role.id=b674fd0a592e4e0cada811bba5ceea4d, scope.project.id=f8b7cfbd340b4753bd898ddc0a462a03, user.id=07914530ee0141de9cc889c157da5a4c) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:33:35.465394 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b53d3d-0171-45fc-9f78-936cf9deff01 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:35.466864 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b53d3d-0171-45fc-9f78-936cf9deff01 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:35.473557 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 89/179] 10.222.0.55 () {56 vars in 1374 bytes} [Wed Aug 24 08:33:35 2022] GET /identity/v3/role_assignments?user.id=07914530ee0141de9cc889c157da5a4c&scope.project.id=f8b7cfbd340b4753bd898ddc0a462a03&role.id=b674fd0a592e4e0cada811bba5ceea4d => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:35.474586 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dc3725-6600-4d0e-809a-a128cdc1f5fc None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:35.475332 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dc3725-6600-4d0e-809a-a128cdc1f5fc None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:35.475617 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dc3725-6600-4d0e-809a-a128cdc1f5fc None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:35.478260 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 90/180] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:33:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:33:35.478394 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dc3725-6600-4d0e-809a-a128cdc1f5fc None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:35.534959 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ac15e9-5f6c-4fd2-a34d-ed3f42f1281e None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:35.578148 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f64fade-47e1-47e9-be6a-ed31c512872d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:35.578783 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f64fade-47e1-47e9-be6a-ed31c512872d None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:35.579076 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f64fade-47e1-47e9-be6a-ed31c512872d None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:35.579390 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f64fade-47e1-47e9-be6a-ed31c512872d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:35.642948 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dc3725-6600-4d0e-809a-a128cdc1f5fc None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:35.648647 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:35.667282 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f64fade-47e1-47e9-be6a-ed31c512872d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:35.668815 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f64fade-47e1-47e9-be6a-ed31c512872d None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:35.684142 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54397457-3acb-403b-981e-77d6b2cb60bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:35.684813 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54397457-3acb-403b-981e-77d6b2cb60bd None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:35.685065 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54397457-3acb-403b-981e-77d6b2cb60bd None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:35.685330 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54397457-3acb-403b-981e-77d6b2cb60bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:35.688473 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:35.767838 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54397457-3acb-403b-981e-77d6b2cb60bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:35.769426 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54397457-3acb-403b-981e-77d6b2cb60bd None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:35.787387 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 91/181] 10.222.0.55 () {58 vars in 1282 bytes} [Wed Aug 24 08:33:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:33:35.788717 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c1ef37-2d01-4dc0-8fac-24eeb558e196 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:35.789050 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c1ef37-2d01-4dc0-8fac-24eeb558e196 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:35.789331 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c1ef37-2d01-4dc0-8fac-24eeb558e196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:35.807438 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-b7c1ef37-2d01-4dc0-8fac-24eeb558e196 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:33:35.808677 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-b7c1ef37-2d01-4dc0-8fac-24eeb558e196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:35.871077 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 91/182] 10.222.0.55 () {58 vars in 1282 bytes} [Wed Aug 24 08:33:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:33:35.871918 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54397457-3acb-403b-981e-77d6b2cb60bd None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:35.892150 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 92/183] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:33:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:33:35.892150 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cc489c-51fd-4f71-84d0-b066755592eb None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:35.892150 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cc489c-51fd-4f71-84d0-b066755592eb None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:35.892150 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cc489c-51fd-4f71-84d0-b066755592eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:35.904514 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-66cc489c-51fd-4f71-84d0-b066755592eb None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:33:35.904514 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-66cc489c-51fd-4f71-84d0-b066755592eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:35.917689 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:35.922822 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:35.996877 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 93/184] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:33:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:33:35.996877 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cc489c-51fd-4f71-84d0-b066755592eb None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:36.033136 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54397457-3acb-403b-981e-77d6b2cb60bd None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:36.089012 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f0852f-90f7-457b-a2e2-7cc7a9dac93e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:36.089650 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f0852f-90f7-457b-a2e2-7cc7a9dac93e None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:36.089923 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f0852f-90f7-457b-a2e2-7cc7a9dac93e None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:36.090630 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f0852f-90f7-457b-a2e2-7cc7a9dac93e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:36.104501 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f0852f-90f7-457b-a2e2-7cc7a9dac93e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:33:36.111252 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-d3f0852f-90f7-457b-a2e2-7cc7a9dac93e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:33:36.111252 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:33:36.111252 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:33:36.111252 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:33:36.111252 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:33:36.111252 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:33:36.111252 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:33:36.111252 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:33:36.111252 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:33:36.111252 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:33:36.111252 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:33:36.111252 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:33:36.111252 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:33:36.111252 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:33:36.111252 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:33:36.111252 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:33:36.112151 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:33:36.112151 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:33:36.112151 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:33:36.112151 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:33:36.112151 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:33:36.112151 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:33:36.112151 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:33:36.112151 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:33:36.112151 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:33:36.112151 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:33:36.112151 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:33:36.112151 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:33:36.112151 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:33:36.112151 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:33:36.112151 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:33:36.112151 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:33:36.113055 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:33:36.113055 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:33:36.113055 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:33:36.113055 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:33:36.113055 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:33:36.113055 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:33:36.113055 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:33:36.113055 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:33:36.113055 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:33:36.113055 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:33:36.113055 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:33:36.113055 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:33:36.113648 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 92/185] 10.222.0.55 () {56 vars in 1096 bytes} [Wed Aug 24 08:33:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:33:36.114211 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f0852f-90f7-457b-a2e2-7cc7a9dac93e None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:36.130333 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cc489c-51fd-4f71-84d0-b066755592eb None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:36.170881 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef1aaf6-691a-42fd-84e9-3b42168245b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:36.171595 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef1aaf6-691a-42fd-84e9-3b42168245b7 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:36.171864 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef1aaf6-691a-42fd-84e9-3b42168245b7 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:36.172256 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef1aaf6-691a-42fd-84e9-3b42168245b7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:36.183537 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef1aaf6-691a-42fd-84e9-3b42168245b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:33:36.188746 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-4ef1aaf6-691a-42fd-84e9-3b42168245b7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 08:33:36.188746 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:33:36.188746 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:33:36.188746 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:33:36.188746 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:33:36.188746 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:33:36.188746 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:33:36.188746 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:33:36.188746 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:33:36.188746 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:33:36.188746 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:33:36.188746 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:33:36.188746 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:33:36.188746 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:33:36.188746 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:33:36.188746 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:33:36.189678 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:33:36.189678 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:33:36.189678 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:33:36.189678 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:33:36.189678 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:33:36.189678 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:33:36.189678 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:33:36.189678 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:33:36.189678 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:33:36.189678 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:33:36.189678 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:33:36.189678 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:33:36.189678 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:33:36.189678 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:33:36.189678 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:33:36.189678 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:33:36.190490 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:33:36.190490 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:33:36.190490 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:33:36.190490 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:33:36.190490 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:33:36.190490 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:33:36.190490 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:33:36.190490 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:33:36.190490 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:33:36.190490 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:33:36.190490 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 08:33:36.190490 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:33:36.190490 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 94/186] 10.222.0.55 () {56 vars in 1099 bytes} [Wed Aug 24 08:33:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:33:36.191156 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef1aaf6-691a-42fd-84e9-3b42168245b7 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:36.203982 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:36.227125 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:36.233894 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f0852f-90f7-457b-a2e2-7cc7a9dac93e None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:36.273179 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f90159a-76f3-4bb8-bcfc-936e1cdd3a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:36.273914 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f90159a-76f3-4bb8-bcfc-936e1cdd3a02 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:36.274189 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f90159a-76f3-4bb8-bcfc-936e1cdd3a02 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:36.274458 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f90159a-76f3-4bb8-bcfc-936e1cdd3a02 None admin] PATH_INFO: `/v3/projects/abc83224889e4e0f8d1675a2cbc7657b` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:36.275664 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f90159a-76f3-4bb8-bcfc-936e1cdd3a02 None admin] RBAC: Authorizing `identity:get_project(project_id=abc83224889e4e0f8d1675a2cbc7657b)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:36.277155 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f90159a-76f3-4bb8-bcfc-936e1cdd3a02 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:36.279028 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 93/187] 10.222.0.55 () {56 vars in 1186 bytes} [Wed Aug 24 08:33:36 2022] GET /identity/v3/projects/abc83224889e4e0f8d1675a2cbc7657b => generated 312 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:36.284411 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f90159a-76f3-4bb8-bcfc-936e1cdd3a02 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:36.303042 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef1aaf6-691a-42fd-84e9-3b42168245b7 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:36.337622 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5409ee96-30a4-4843-9e3e-1197e466372a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:36.338245 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5409ee96-30a4-4843-9e3e-1197e466372a None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:36.338518 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5409ee96-30a4-4843-9e3e-1197e466372a None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:36.338785 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5409ee96-30a4-4843-9e3e-1197e466372a None admin] PATH_INFO: `/v3/projects/f8b7cfbd340b4753bd898ddc0a462a03` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:36.339897 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5409ee96-30a4-4843-9e3e-1197e466372a None admin] RBAC: Authorizing `identity:get_project(project_id=f8b7cfbd340b4753bd898ddc0a462a03)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:36.341382 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5409ee96-30a4-4843-9e3e-1197e466372a None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:36.342989 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 95/188] 10.222.0.55 () {56 vars in 1186 bytes} [Wed Aug 24 08:33:36 2022] GET /identity/v3/projects/f8b7cfbd340b4753bd898ddc0a462a03 => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:36.343744 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5409ee96-30a4-4843-9e3e-1197e466372a None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:36.387273 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f90159a-76f3-4bb8-bcfc-936e1cdd3a02 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:36.417833 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:36.421065 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9188340c-952b-4319-8d85-9eb0ba89d8b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:36.421627 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9188340c-952b-4319-8d85-9eb0ba89d8b3 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:36.421908 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9188340c-952b-4319-8d85-9eb0ba89d8b3 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:36.422183 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9188340c-952b-4319-8d85-9eb0ba89d8b3 None admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:36.422854 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9188340c-952b-4319-8d85-9eb0ba89d8b3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:33:36.423159 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9188340c-952b-4319-8d85-9eb0ba89d8b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:36.424580 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9188340c-952b-4319-8d85-9eb0ba89d8b3 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:36.434331 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 94/189] 10.222.0.55 () {56 vars in 1099 bytes} [Wed Aug 24 08:33:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:36.435404 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9188340c-952b-4319-8d85-9eb0ba89d8b3 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:36.448587 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5409ee96-30a4-4843-9e3e-1197e466372a None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:36.463634 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:36.482992 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3195d400-e825-470f-9af4-79108923c895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:36.483617 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3195d400-e825-470f-9af4-79108923c895 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:36.483898 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3195d400-e825-470f-9af4-79108923c895 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:36.484166 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3195d400-e825-470f-9af4-79108923c895 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:36.495094 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3195d400-e825-470f-9af4-79108923c895 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:33:36.500007 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-3195d400-e825-470f-9af4-79108923c895 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 08:33:36.500007 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:33:36.500007 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:33:36.500007 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:33:36.500007 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:33:36.500007 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:33:36.500007 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:33:36.500007 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:33:36.500007 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:33:36.500007 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:33:36.500007 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:33:36.500007 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:33:36.500007 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:33:36.500007 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:33:36.500007 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:33:36.500007 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:33:36.500946 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:33:36.500946 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:33:36.500946 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:33:36.500946 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:33:36.500946 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:33:36.500946 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:33:36.500946 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:33:36.500946 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:33:36.500946 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:33:36.500946 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:33:36.500946 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:33:36.500946 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:33:36.500946 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:33:36.500946 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:33:36.500946 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:33:36.500946 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:33:36.501717 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:33:36.501717 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:33:36.501717 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:33:36.501717 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:33:36.501717 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:33:36.501717 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:33:36.501717 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:33:36.501717 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:33:36.501717 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:33:36.501717 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:33:36.501717 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 08:33:36.501717 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:33:36.501717 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 96/190] 10.222.0.55 () {56 vars in 1114 bytes} [Wed Aug 24 08:33:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:33:36.502383 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3195d400-e825-470f-9af4-79108923c895 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:36.537490 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9188340c-952b-4319-8d85-9eb0ba89d8b3 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:36.571755 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c695ea0f-3da4-4879-a0d1-d7d9cce471e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:36.572364 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c695ea0f-3da4-4879-a0d1-d7d9cce471e0 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:36.572673 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c695ea0f-3da4-4879-a0d1-d7d9cce471e0 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:36.572947 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c695ea0f-3da4-4879-a0d1-d7d9cce471e0 None admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:36.573616 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c695ea0f-3da4-4879-a0d1-d7d9cce471e0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:33:36.573917 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c695ea0f-3da4-4879-a0d1-d7d9cce471e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:36.575283 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c695ea0f-3da4-4879-a0d1-d7d9cce471e0 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:36.583303 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 95/191] 10.222.0.55 () {56 vars in 1101 bytes} [Wed Aug 24 08:33:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:36.584095 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c695ea0f-3da4-4879-a0d1-d7d9cce471e0 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:36.607494 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3195d400-e825-470f-9af4-79108923c895 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:36.643479 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96f3167-11ac-4922-9461-6ec602b4c559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:36.644132 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96f3167-11ac-4922-9461-6ec602b4c559 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:36.644408 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96f3167-11ac-4922-9461-6ec602b4c559 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:36.644717 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96f3167-11ac-4922-9461-6ec602b4c559 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:36.645448 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96f3167-11ac-4922-9461-6ec602b4c559 None admin] RBAC: Adding query filter params (role.id=b674fd0a592e4e0cada811bba5ceea4d, scope.project.id=abc83224889e4e0f8d1675a2cbc7657b, user.id=07914530ee0141de9cc889c157da5a4c) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:33:36.645755 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96f3167-11ac-4922-9461-6ec602b4c559 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:36.647181 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96f3167-11ac-4922-9461-6ec602b4c559 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:36.648960 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:36.653338 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 97/192] 10.222.0.55 () {56 vars in 1374 bytes} [Wed Aug 24 08:33:36 2022] GET /identity/v3/role_assignments?user.id=07914530ee0141de9cc889c157da5a4c&scope.project.id=abc83224889e4e0f8d1675a2cbc7657b&role.id=b674fd0a592e4e0cada811bba5ceea4d => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:36.657452 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96f3167-11ac-4922-9461-6ec602b4c559 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:36.670587 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:36.689798 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c695ea0f-3da4-4879-a0d1-d7d9cce471e0 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:36.729851 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13f9a9b-c50c-4a3c-bb99-9faa157f62dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:36.730507 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13f9a9b-c50c-4a3c-bb99-9faa157f62dc None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:36.730784 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13f9a9b-c50c-4a3c-bb99-9faa157f62dc None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:36.731050 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13f9a9b-c50c-4a3c-bb99-9faa157f62dc None admin] PATH_INFO: `/v3/domains/default` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:36.732241 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13f9a9b-c50c-4a3c-bb99-9faa157f62dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:36.733778 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13f9a9b-c50c-4a3c-bb99-9faa157f62dc None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:36.740062 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 96/193] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:33:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:36.741012 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13f9a9b-c50c-4a3c-bb99-9faa157f62dc None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:36.779165 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96f3167-11ac-4922-9461-6ec602b4c559 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:36.822551 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fcb416-1417-4be3-9744-21f2a79df9ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:36.823166 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fcb416-1417-4be3-9744-21f2a79df9ec None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:36.823449 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fcb416-1417-4be3-9744-21f2a79df9ec None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:36.823721 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fcb416-1417-4be3-9744-21f2a79df9ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:36.824458 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fcb416-1417-4be3-9744-21f2a79df9ec None admin] RBAC: Adding query filter params (role.id=8b5ad96041894155b0138a676ead607e, scope.project.id=f8b7cfbd340b4753bd898ddc0a462a03, user.id=07914530ee0141de9cc889c157da5a4c) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:33:36.825462 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fcb416-1417-4be3-9744-21f2a79df9ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:36.826303 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fcb416-1417-4be3-9744-21f2a79df9ec None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:36.833936 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 98/194] 10.222.0.55 () {56 vars in 1374 bytes} [Wed Aug 24 08:33:36 2022] GET /identity/v3/role_assignments?user.id=07914530ee0141de9cc889c157da5a4c&scope.project.id=f8b7cfbd340b4753bd898ddc0a462a03&role.id=8b5ad96041894155b0138a676ead607e => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:36.834751 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fcb416-1417-4be3-9744-21f2a79df9ec None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:36.866486 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13f9a9b-c50c-4a3c-bb99-9faa157f62dc None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:36.919038 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd251c8-dd54-4778-9b9c-ba17afadba2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:36.920250 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd251c8-dd54-4778-9b9c-ba17afadba2e None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:36.920250 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd251c8-dd54-4778-9b9c-ba17afadba2e None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:36.920644 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd251c8-dd54-4778-9b9c-ba17afadba2e None admin] PATH_INFO: `/v3/users/5e3ac3a673394ce79418e671768a7458` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:36.929546 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:36.938330 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd251c8-dd54-4778-9b9c-ba17afadba2e None admin] RBAC: Authorizing `identity:get_user(user_id=5e3ac3a673394ce79418e671768a7458)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:36.946176 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd251c8-dd54-4778-9b9c-ba17afadba2e None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:36.946176 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 97/195] 10.222.0.55 () {56 vars in 1177 bytes} [Wed Aug 24 08:33:36 2022] GET /identity/v3/users/5e3ac3a673394ce79418e671768a7458 => generated 283 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:36.946176 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd251c8-dd54-4778-9b9c-ba17afadba2e None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:36.960076 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:36.985341 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fcb416-1417-4be3-9744-21f2a79df9ec None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:37.034979 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1784be-a278-4798-bdc1-a04a316b7699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:37.035672 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1784be-a278-4798-bdc1-a04a316b7699 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:37.035956 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1784be-a278-4798-bdc1-a04a316b7699 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:37.036229 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1784be-a278-4798-bdc1-a04a316b7699 None admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:37.036968 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1784be-a278-4798-bdc1-a04a316b7699 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:33:37.037282 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1784be-a278-4798-bdc1-a04a316b7699 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:37.038717 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1784be-a278-4798-bdc1-a04a316b7699 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:37.049799 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 99/196] 10.222.0.55 () {56 vars in 1111 bytes} [Wed Aug 24 08:33:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:37.050750 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1784be-a278-4798-bdc1-a04a316b7699 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:37.070111 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd251c8-dd54-4778-9b9c-ba17afadba2e None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:37.107026 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192061ee-45a6-4885-8192-ea1a2ee68ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:37.107679 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192061ee-45a6-4885-8192-ea1a2ee68ced None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:37.107954 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192061ee-45a6-4885-8192-ea1a2ee68ced None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:37.108226 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192061ee-45a6-4885-8192-ea1a2ee68ced None admin] PATH_INFO: `/v3/users/5b29b7c0626f4c3da644d63cf7510055` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:37.109468 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192061ee-45a6-4885-8192-ea1a2ee68ced None admin] RBAC: Authorizing `identity:get_user(user_id=5b29b7c0626f4c3da644d63cf7510055)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:37.110938 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192061ee-45a6-4885-8192-ea1a2ee68ced None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:37.112584 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 98/197] 10.222.0.55 () {56 vars in 1177 bytes} [Wed Aug 24 08:33:36 2022] GET /identity/v3/users/5b29b7c0626f4c3da644d63cf7510055 => generated 289 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:37.113393 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192061ee-45a6-4885-8192-ea1a2ee68ced None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:37.169815 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1784be-a278-4798-bdc1-a04a316b7699 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:37.195369 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:37.201499 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:37.208026 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04963af-310f-42ca-b0ab-6fcea26f0ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:37.208707 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04963af-310f-42ca-b0ab-6fcea26f0ce6 None admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:37.211045 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04963af-310f-42ca-b0ab-6fcea26f0ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:37.211045 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04963af-310f-42ca-b0ab-6fcea26f0ce6 None admin] PATH_INFO: `/v3/users` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:37.211045 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04963af-310f-42ca-b0ab-6fcea26f0ce6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:37.211469 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04963af-310f-42ca-b0ab-6fcea26f0ce6 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:37.222803 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192061ee-45a6-4885-8192-ea1a2ee68ced None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:37.257372 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19ee518-bfd4-4b67-b221-8978df2b8c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:37.257999 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19ee518-bfd4-4b67-b221-8978df2b8c56 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:37.258286 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19ee518-bfd4-4b67-b221-8978df2b8c56 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:37.258549 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19ee518-bfd4-4b67-b221-8978df2b8c56 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:37.291153 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 100/198] 10.222.0.55 () {60 vars in 1132 bytes} [Wed Aug 24 08:33:37 2022] POST /identity/v3/users => generated 297 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:33:37.292509 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04963af-310f-42ca-b0ab-6fcea26f0ce6 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:37.354968 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19ee518-bfd4-4b67-b221-8978df2b8c56 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:37.357559 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19ee518-bfd4-4b67-b221-8978df2b8c56 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:37.416100 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04963af-310f-42ca-b0ab-6fcea26f0ce6 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:37.455366 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76909fc-a99e-4d30-83fd-405b41026333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:37.455969 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76909fc-a99e-4d30-83fd-405b41026333 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:37.456202 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76909fc-a99e-4d30-83fd-405b41026333 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:37.456469 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76909fc-a99e-4d30-83fd-405b41026333 None admin] PATH_INFO: `/v3/users/5e3ac3a673394ce79418e671768a7458` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:37.457651 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76909fc-a99e-4d30-83fd-405b41026333 None admin] RBAC: Authorizing `identity:get_user(user_id=5e3ac3a673394ce79418e671768a7458)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:37.459101 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76909fc-a99e-4d30-83fd-405b41026333 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:37.460358 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:37.460804 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 101/199] 10.222.0.55 () {56 vars in 1177 bytes} [Wed Aug 24 08:33:37 2022] GET /identity/v3/users/5e3ac3a673394ce79418e671768a7458 => generated 283 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:37.461482 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76909fc-a99e-4d30-83fd-405b41026333 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:37.482653 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 99/200] 10.222.0.55 () {58 vars in 1282 bytes} [Wed Aug 24 08:33:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:33:37.483599 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19ee518-bfd4-4b67-b221-8978df2b8c56 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:37.519601 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:37.585434 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76909fc-a99e-4d30-83fd-405b41026333 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:37.612112 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19ee518-bfd4-4b67-b221-8978df2b8c56 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:37.625626 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a30143-43f0-4512-96a4-3cef9b9ade42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:37.626253 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a30143-43f0-4512-96a4-3cef9b9ade42 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:37.626533 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a30143-43f0-4512-96a4-3cef9b9ade42 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:37.626808 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a30143-43f0-4512-96a4-3cef9b9ade42 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:37.653753 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0805da43-cb7c-466b-93c5-5bc54a811390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:37.654439 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0805da43-cb7c-466b-93c5-5bc54a811390 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:37.654718 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0805da43-cb7c-466b-93c5-5bc54a811390 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:37.654993 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0805da43-cb7c-466b-93c5-5bc54a811390 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:37.711540 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a30143-43f0-4512-96a4-3cef9b9ade42 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:37.712946 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a30143-43f0-4512-96a4-3cef9b9ade42 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:37.718088 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:37.744877 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0805da43-cb7c-466b-93c5-5bc54a811390 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:37.747625 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0805da43-cb7c-466b-93c5-5bc54a811390 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:37.773880 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:37.830709 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 102/201] 10.222.0.55 () {58 vars in 1282 bytes} [Wed Aug 24 08:33:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:33:37.831527 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a30143-43f0-4512-96a4-3cef9b9ade42 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:37.895008 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 100/202] 10.222.0.55 () {58 vars in 1282 bytes} [Wed Aug 24 08:33:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:33:37.896257 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0805da43-cb7c-466b-93c5-5bc54a811390 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:37.942904 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a30143-43f0-4512-96a4-3cef9b9ade42 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:37.980087 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a77ce6-328d-4fb0-bc0a-b3d37fdf70b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:37.980674 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a77ce6-328d-4fb0-bc0a-b3d37fdf70b0 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:37.980959 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a77ce6-328d-4fb0-bc0a-b3d37fdf70b0 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:37.981236 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a77ce6-328d-4fb0-bc0a-b3d37fdf70b0 None admin] PATH_INFO: `/v3/projects/61247a1049b846b79052f42b0d525259` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:37.982569 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a77ce6-328d-4fb0-bc0a-b3d37fdf70b0 None admin] RBAC: Authorizing `identity:get_project(project_id=61247a1049b846b79052f42b0d525259)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:37.984685 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a77ce6-328d-4fb0-bc0a-b3d37fdf70b0 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:37.987197 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 103/203] 10.222.0.55 () {56 vars in 1186 bytes} [Wed Aug 24 08:33:37 2022] GET /identity/v3/projects/61247a1049b846b79052f42b0d525259 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:37.988814 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a77ce6-328d-4fb0-bc0a-b3d37fdf70b0 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:38.002513 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:38.020927 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:38.027782 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0805da43-cb7c-466b-93c5-5bc54a811390 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:38.068292 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f6e52f-367c-439c-8331-669844326c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:38.069009 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f6e52f-367c-439c-8331-669844326c56 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:38.069294 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f6e52f-367c-439c-8331-669844326c56 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:38.069572 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f6e52f-367c-439c-8331-669844326c56 None admin] PATH_INFO: `/v3/projects/f8b7cfbd340b4753bd898ddc0a462a03` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:38.071127 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f6e52f-367c-439c-8331-669844326c56 None admin] RBAC: Authorizing `identity:get_project(project_id=f8b7cfbd340b4753bd898ddc0a462a03)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:38.072942 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f6e52f-367c-439c-8331-669844326c56 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:38.074614 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 101/204] 10.222.0.55 () {56 vars in 1186 bytes} [Wed Aug 24 08:33:37 2022] GET /identity/v3/projects/f8b7cfbd340b4753bd898ddc0a462a03 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:38.075740 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f6e52f-367c-439c-8331-669844326c56 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:38.115362 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a77ce6-328d-4fb0-bc0a-b3d37fdf70b0 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:38.162644 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e93f2f3-a4cc-4f49-b34a-991698038020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:38.163291 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e93f2f3-a4cc-4f49-b34a-991698038020 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:38.163571 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e93f2f3-a4cc-4f49-b34a-991698038020 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:38.163841 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e93f2f3-a4cc-4f49-b34a-991698038020 None admin] PATH_INFO: `/v3/projects/61247a1049b846b79052f42b0d525259` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:38.169853 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e93f2f3-a4cc-4f49-b34a-991698038020 None admin] RBAC: Authorizing `identity:get_project(project_id=61247a1049b846b79052f42b0d525259)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:38.172147 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e93f2f3-a4cc-4f49-b34a-991698038020 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:38.175159 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 104/205] 10.222.0.55 () {56 vars in 1186 bytes} [Wed Aug 24 08:33:37 2022] GET /identity/v3/projects/61247a1049b846b79052f42b0d525259 => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:38.176407 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e93f2f3-a4cc-4f49-b34a-991698038020 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:38.196198 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f6e52f-367c-439c-8331-669844326c56 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:38.241399 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd5eba4-7a87-4798-81df-0dc0f7de3978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:38.241930 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd5eba4-7a87-4798-81df-0dc0f7de3978 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:38.242212 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd5eba4-7a87-4798-81df-0dc0f7de3978 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:38.242483 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd5eba4-7a87-4798-81df-0dc0f7de3978 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:38.243212 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd5eba4-7a87-4798-81df-0dc0f7de3978 None admin] RBAC: Adding query filter params (role.id=b870cd1acaa744e1ae1d58b9ac283821, scope.project.id=61247a1049b846b79052f42b0d525259, user.id=5e3ac3a673394ce79418e671768a7458) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:33:38.243552 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd5eba4-7a87-4798-81df-0dc0f7de3978 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:38.245073 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd5eba4-7a87-4798-81df-0dc0f7de3978 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:38.251391 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 102/206] 10.222.0.55 () {56 vars in 1374 bytes} [Wed Aug 24 08:33:38 2022] GET /identity/v3/role_assignments?user.id=5e3ac3a673394ce79418e671768a7458&scope.project.id=61247a1049b846b79052f42b0d525259&role.id=b870cd1acaa744e1ae1d58b9ac283821 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:38.252337 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd5eba4-7a87-4798-81df-0dc0f7de3978 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:38.268240 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:38.275148 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:38.293825 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e93f2f3-a4cc-4f49-b34a-991698038020 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:38.334189 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd3d6d8-74d1-40c9-a116-a1c65bbd93a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:38.334816 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd3d6d8-74d1-40c9-a116-a1c65bbd93a8 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:38.335092 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd3d6d8-74d1-40c9-a116-a1c65bbd93a8 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:38.335361 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd3d6d8-74d1-40c9-a116-a1c65bbd93a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:38.336194 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd3d6d8-74d1-40c9-a116-a1c65bbd93a8 None admin] RBAC: Adding query filter params (role.id=e308031ba9a84b79bcdf501455ebe0d7, scope.project.id=f8b7cfbd340b4753bd898ddc0a462a03, user.id=5b29b7c0626f4c3da644d63cf7510055) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:33:38.336585 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd3d6d8-74d1-40c9-a116-a1c65bbd93a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:38.338670 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd3d6d8-74d1-40c9-a116-a1c65bbd93a8 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:38.345248 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 105/207] 10.222.0.55 () {56 vars in 1374 bytes} [Wed Aug 24 08:33:38 2022] GET /identity/v3/role_assignments?user.id=5b29b7c0626f4c3da644d63cf7510055&scope.project.id=f8b7cfbd340b4753bd898ddc0a462a03&role.id=e308031ba9a84b79bcdf501455ebe0d7 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:38.373813 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd5eba4-7a87-4798-81df-0dc0f7de3978 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:38.413830 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4075c3-60f7-436d-832d-cf6d806bb45e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:38.414463 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4075c3-60f7-436d-832d-cf6d806bb45e None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:38.414744 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4075c3-60f7-436d-832d-cf6d806bb45e None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:38.415015 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4075c3-60f7-436d-832d-cf6d806bb45e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:38.416477 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4075c3-60f7-436d-832d-cf6d806bb45e None admin] RBAC: Adding query filter params (role.id=8b5ad96041894155b0138a676ead607e, scope.project.id=61247a1049b846b79052f42b0d525259, user.id=5e3ac3a673394ce79418e671768a7458) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:33:38.416840 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4075c3-60f7-436d-832d-cf6d806bb45e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:38.418336 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4075c3-60f7-436d-832d-cf6d806bb45e None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:38.424863 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 103/208] 10.222.0.55 () {56 vars in 1374 bytes} [Wed Aug 24 08:33:38 2022] GET /identity/v3/role_assignments?user.id=5e3ac3a673394ce79418e671768a7458&scope.project.id=61247a1049b846b79052f42b0d525259&role.id=8b5ad96041894155b0138a676ead607e => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:39.499800 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e571cd5-8618-49de-be70-84220894c5d6 None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:39.500310 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e571cd5-8618-49de-be70-84220894c5d6 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:39.500513 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e571cd5-8618-49de-be70-84220894c5d6 None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:39.502383 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 106/209] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:33:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:33:39.508028 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79b3d68-adf8-4161-9bf6-d826ff7c719d None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:39.508463 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79b3d68-adf8-4161-9bf6-d826ff7c719d None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:39.508811 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79b3d68-adf8-4161-9bf6-d826ff7c719d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:39.576080 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9e28a6-a5b2-4748-aabc-bbd5fc3048a7 None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:39.576383 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9e28a6-a5b2-4748-aabc-bbd5fc3048a7 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:39.576714 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9e28a6-a5b2-4748-aabc-bbd5fc3048a7 None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:39.578335 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 107/210] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:33:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:33:39.582216 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-e79b3d68-adf8-4161-9bf6-d826ff7c719d None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:33:39.583604 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef625ea7-215f-4618-b001-5e5e70384588 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:39.583724 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-e79b3d68-adf8-4161-9bf6-d826ff7c719d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:39.583884 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef625ea7-215f-4618-b001-5e5e70384588 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:39.584161 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef625ea7-215f-4618-b001-5e5e70384588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:39.606406 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-ef625ea7-215f-4618-b001-5e5e70384588 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:33:39.608416 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-ef625ea7-215f-4618-b001-5e5e70384588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:39.673535 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 104/211] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:33:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:33:39.698172 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 108/212] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:33:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:33:39.955588 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79b3d68-adf8-4161-9bf6-d826ff7c719d None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:39.965147 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:39.967576 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef625ea7-215f-4618-b001-5e5e70384588 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:40.064942 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:40.081650 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef625ea7-215f-4618-b001-5e5e70384588 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:40.091664 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79b3d68-adf8-4161-9bf6-d826ff7c719d None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:40.128747 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2433f065-faea-4796-92e5-13930d8116d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:40.129569 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2433f065-faea-4796-92e5-13930d8116d5 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:40.129854 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2433f065-faea-4796-92e5-13930d8116d5 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:40.130129 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2433f065-faea-4796-92e5-13930d8116d5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:40.130415 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f885bf5f-b540-4a9b-a18a-6c58960627f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:40.131149 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f885bf5f-b540-4a9b-a18a-6c58960627f2 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:40.131305 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2433f065-faea-4796-92e5-13930d8116d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:40.131611 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f885bf5f-b540-4a9b-a18a-6c58960627f2 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:40.131978 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f885bf5f-b540-4a9b-a18a-6c58960627f2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:40.132891 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2433f065-faea-4796-92e5-13930d8116d5 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:40.136644 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 109/213] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:33:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:33:40.140339 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2433f065-faea-4796-92e5-13930d8116d5 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:40.151025 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f885bf5f-b540-4a9b-a18a-6c58960627f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:33:40.156133 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-f885bf5f-b540-4a9b-a18a-6c58960627f2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:33:40.156133 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:33:40.156133 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:33:40.156133 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:33:40.156133 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:33:40.156133 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:33:40.156133 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:33:40.156133 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:33:40.156133 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:33:40.156133 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:33:40.156133 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:33:40.156133 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:33:40.156133 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:33:40.156133 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:33:40.156133 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:33:40.156133 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:33:40.157248 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:33:40.157248 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:33:40.157248 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:33:40.157248 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:33:40.157248 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:33:40.157248 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:33:40.157248 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:33:40.157248 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:33:40.157248 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:33:40.157248 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:33:40.157248 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:33:40.157248 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:33:40.157248 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:33:40.157248 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:33:40.157248 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:33:40.157248 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:33:40.158039 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:33:40.158039 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:33:40.158039 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:33:40.158039 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:33:40.158039 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:33:40.158039 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:33:40.158039 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:33:40.158039 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:33:40.158039 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:33:40.158039 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:33:40.158039 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:33:40.158039 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:33:40.162561 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 105/214] 10.222.0.55 () {56 vars in 1099 bytes} [Wed Aug 24 08:33:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:33:40.164090 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f885bf5f-b540-4a9b-a18a-6c58960627f2 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:40.246287 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:40.283049 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f885bf5f-b540-4a9b-a18a-6c58960627f2 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:40.284423 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2433f065-faea-4796-92e5-13930d8116d5 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:40.329450 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39172dbf-e04a-463f-ba94-d91582a75e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:40.330072 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39172dbf-e04a-463f-ba94-d91582a75e6c None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:40.330358 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39172dbf-e04a-463f-ba94-d91582a75e6c None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:40.330635 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39172dbf-e04a-463f-ba94-d91582a75e6c None admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:40.331309 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39172dbf-e04a-463f-ba94-d91582a75e6c None admin] RBAC: Adding query filter params (name=member) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:33:40.331615 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39172dbf-e04a-463f-ba94-d91582a75e6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:40.333055 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39172dbf-e04a-463f-ba94-d91582a75e6c None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:40.339200 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a9e46a-d675-42ed-8ae1-06fc5d68c4ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:40.339844 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a9e46a-d675-42ed-8ae1-06fc5d68c4ca None admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:40.340125 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a9e46a-d675-42ed-8ae1-06fc5d68c4ca None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:40.340423 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a9e46a-d675-42ed-8ae1-06fc5d68c4ca None admin] PATH_INFO: `/v3/users` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:40.341146 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a9e46a-d675-42ed-8ae1-06fc5d68c4ca None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:40.341730 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 106/215] 10.222.0.55 () {56 vars in 1101 bytes} [Wed Aug 24 08:33:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:33:40.342764 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6c3978-3af4-4d00-94eb-27cbf1afda26 None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:40.343051 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6c3978-3af4-4d00-94eb-27cbf1afda26 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:40.343327 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6c3978-3af4-4d00-94eb-27cbf1afda26 None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:40.348826 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 107/216] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:33:40 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:33:40.348950 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a9e46a-d675-42ed-8ae1-06fc5d68c4ca None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:40.349943 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238f1978-87f0-4bde-a82e-4b71be92fb3e None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:40.350223 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238f1978-87f0-4bde-a82e-4b71be92fb3e None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:40.350496 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238f1978-87f0-4bde-a82e-4b71be92fb3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:40.371135 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-238f1978-87f0-4bde-a82e-4b71be92fb3e None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:33:40.372336 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-238f1978-87f0-4bde-a82e-4b71be92fb3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:40.464527 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 110/217] 10.222.0.55 () {60 vars in 1132 bytes} [Wed Aug 24 08:33:40 2022] POST /identity/v3/users => generated 297 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:33:40.465466 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a9e46a-d675-42ed-8ae1-06fc5d68c4ca None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:40.468233 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:40.476444 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 108/218] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:33:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:33:40.478670 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac84b6a-ff5c-4608-a73e-0a195f249844 None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:40.479235 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac84b6a-ff5c-4608-a73e-0a195f249844 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:40.479695 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac84b6a-ff5c-4608-a73e-0a195f249844 None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:40.481326 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 109/219] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:33:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:33:40.487096 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2687ee6c-7d86-4450-9411-118a2836742a None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:40.487393 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2687ee6c-7d86-4450-9411-118a2836742a None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:40.487690 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2687ee6c-7d86-4450-9411-118a2836742a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:40.507796 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-2687ee6c-7d86-4450-9411-118a2836742a None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:33:40.509028 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-2687ee6c-7d86-4450-9411-118a2836742a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:40.599474 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:40.620689 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a9e46a-d675-42ed-8ae1-06fc5d68c4ca None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:40.621992 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 110/220] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:33:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 08:33:40.673774 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0bb64f-205b-489e-966d-d3b4af3ca84c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:40.674441 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0bb64f-205b-489e-966d-d3b4af3ca84c None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:40.674725 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0bb64f-205b-489e-966d-d3b4af3ca84c None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:40.675002 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0bb64f-205b-489e-966d-d3b4af3ca84c None admin] PATH_INFO: `/v3/users/10ab6b64fbda471a8f7756070346fe3b` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:40.693594 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0bb64f-205b-489e-966d-d3b4af3ca84c None admin] RBAC: Authorizing `identity:get_user(user_id=10ab6b64fbda471a8f7756070346fe3b)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:40.695077 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0bb64f-205b-489e-966d-d3b4af3ca84c None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:40.697353 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 111/221] 10.222.0.55 () {56 vars in 1177 bytes} [Wed Aug 24 08:33:40 2022] GET /identity/v3/users/10ab6b64fbda471a8f7756070346fe3b => generated 297 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:40.703282 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2687ee6c-7d86-4450-9411-118a2836742a None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:40.755426 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0bb64f-205b-489e-966d-d3b4af3ca84c None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:40.827635 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2687ee6c-7d86-4450-9411-118a2836742a None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:40.838733 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:40.874893 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f7074e-4610-4787-ad5d-fe1ffc844779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:40.875540 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f7074e-4610-4787-ad5d-fe1ffc844779 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:40.875809 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f7074e-4610-4787-ad5d-fe1ffc844779 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:40.876079 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f7074e-4610-4787-ad5d-fe1ffc844779 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:40.895535 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0bb64f-205b-489e-966d-d3b4af3ca84c None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:40.937963 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16385a77-ba5a-4c94-8d69-77b2bf1a7de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:40.938728 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16385a77-ba5a-4c94-8d69-77b2bf1a7de8 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:40.939005 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16385a77-ba5a-4c94-8d69-77b2bf1a7de8 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:40.939277 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16385a77-ba5a-4c94-8d69-77b2bf1a7de8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:40.945201 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:40.953642 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16385a77-ba5a-4c94-8d69-77b2bf1a7de8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:33:40.958968 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-16385a77-ba5a-4c94-8d69-77b2bf1a7de8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:33:40.958968 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:33:40.958968 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:33:40.958968 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:33:40.958968 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:33:40.958968 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:33:40.958968 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:33:40.958968 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:33:40.958968 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:33:40.958968 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:33:40.958968 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:33:40.958968 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:33:40.958968 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:33:40.958968 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:33:40.958968 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:33:40.958968 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:33:40.959895 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:33:40.959895 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:33:40.959895 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:33:40.959895 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:33:40.959895 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:33:40.959895 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:33:40.959895 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:33:40.959895 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:33:40.959895 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:33:40.959895 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:33:40.959895 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:33:40.959895 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:33:40.959895 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:33:40.959895 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:33:40.959895 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:33:40.959895 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:33:40.960728 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:33:40.960728 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:33:40.960728 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:33:40.960728 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:33:40.960728 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:33:40.960728 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:33:40.960728 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:33:40.960728 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:33:40.960728 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:33:40.960728 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:33:40.960728 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:33:40.960728 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:33:40.960728 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 112/222] 10.222.0.55 () {56 vars in 1096 bytes} [Wed Aug 24 08:33:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:33:40.961388 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16385a77-ba5a-4c94-8d69-77b2bf1a7de8 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:40.973082 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f7074e-4610-4787-ad5d-fe1ffc844779 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:40.974583 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f7074e-4610-4787-ad5d-fe1ffc844779 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:41.094905 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16385a77-ba5a-4c94-8d69-77b2bf1a7de8 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:41.105834 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 111/223] 10.222.0.55 () {58 vars in 1282 bytes} [Wed Aug 24 08:33:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:33:41.107041 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f7074e-4610-4787-ad5d-fe1ffc844779 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:41.121751 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:41.142330 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b045ac2-e5ce-46e1-bba0-1258e8ee4912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:41.142982 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b045ac2-e5ce-46e1-bba0-1258e8ee4912 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:41.143262 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b045ac2-e5ce-46e1-bba0-1258e8ee4912 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:41.143533 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b045ac2-e5ce-46e1-bba0-1258e8ee4912 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:41.159649 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b045ac2-e5ce-46e1-bba0-1258e8ee4912 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:33:41.166453 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-7b045ac2-e5ce-46e1-bba0-1258e8ee4912 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 08:33:41.166453 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:33:41.166453 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:33:41.166453 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:33:41.166453 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:33:41.166453 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:33:41.166453 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:33:41.166453 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:33:41.166453 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:33:41.166453 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:33:41.166453 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:33:41.166453 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:33:41.166453 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:33:41.166453 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:33:41.166453 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:33:41.166453 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:33:41.167343 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:33:41.167343 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:33:41.167343 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:33:41.167343 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:33:41.167343 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:33:41.167343 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:33:41.167343 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:33:41.167343 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:33:41.167343 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:33:41.167343 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:33:41.167343 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:33:41.167343 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:33:41.167343 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:33:41.167343 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:33:41.167343 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:33:41.167343 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:33:41.168181 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:33:41.168181 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:33:41.168181 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:33:41.168181 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:33:41.168181 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:33:41.168181 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:33:41.168181 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:33:41.168181 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:33:41.168181 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:33:41.168181 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:33:41.168181 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 08:33:41.168181 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:33:41.168181 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 113/224] 10.222.0.55 () {56 vars in 1114 bytes} [Wed Aug 24 08:33:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:33:41.168877 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b045ac2-e5ce-46e1-bba0-1258e8ee4912 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:41.229989 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:41.247034 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f7074e-4610-4787-ad5d-fe1ffc844779 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:41.287354 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eccb8b5-6582-45f9-a765-7f1ddc23f787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:41.287960 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eccb8b5-6582-45f9-a765-7f1ddc23f787 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:41.288231 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eccb8b5-6582-45f9-a765-7f1ddc23f787 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:41.288496 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eccb8b5-6582-45f9-a765-7f1ddc23f787 None admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:41.289202 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eccb8b5-6582-45f9-a765-7f1ddc23f787 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:33:41.289503 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eccb8b5-6582-45f9-a765-7f1ddc23f787 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:41.290896 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eccb8b5-6582-45f9-a765-7f1ddc23f787 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:41.293051 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b045ac2-e5ce-46e1-bba0-1258e8ee4912 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:41.299475 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 112/225] 10.222.0.55 () {56 vars in 1099 bytes} [Wed Aug 24 08:33:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:41.300130 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eccb8b5-6582-45f9-a765-7f1ddc23f787 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:41.337180 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100487ab-b729-46da-9f2b-da852db1ea85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:41.337849 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100487ab-b729-46da-9f2b-da852db1ea85 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:41.338120 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100487ab-b729-46da-9f2b-da852db1ea85 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:41.338418 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100487ab-b729-46da-9f2b-da852db1ea85 None admin] PATH_INFO: `/v3/projects/61247a1049b846b79052f42b0d525259` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:41.341227 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100487ab-b729-46da-9f2b-da852db1ea85 None admin] RBAC: Authorizing `identity:get_project(project_id=61247a1049b846b79052f42b0d525259)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:41.343466 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100487ab-b729-46da-9f2b-da852db1ea85 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:41.345924 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 114/226] 10.222.0.55 () {56 vars in 1186 bytes} [Wed Aug 24 08:33:41 2022] GET /identity/v3/projects/61247a1049b846b79052f42b0d525259 => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:41.347780 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100487ab-b729-46da-9f2b-da852db1ea85 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:41.409170 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eccb8b5-6582-45f9-a765-7f1ddc23f787 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:41.419424 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:41.446411 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58914467-3ae5-48f9-afe3-b3fc4d77cb1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:41.447611 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58914467-3ae5-48f9-afe3-b3fc4d77cb1e None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:41.447896 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58914467-3ae5-48f9-afe3-b3fc4d77cb1e None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:41.448167 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58914467-3ae5-48f9-afe3-b3fc4d77cb1e None admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:41.448874 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58914467-3ae5-48f9-afe3-b3fc4d77cb1e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:33:41.449172 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58914467-3ae5-48f9-afe3-b3fc4d77cb1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:41.451399 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58914467-3ae5-48f9-afe3-b3fc4d77cb1e None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:41.460451 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 113/227] 10.222.0.55 () {56 vars in 1111 bytes} [Wed Aug 24 08:33:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:41.461134 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58914467-3ae5-48f9-afe3-b3fc4d77cb1e None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:41.481814 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:41.491467 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100487ab-b729-46da-9f2b-da852db1ea85 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:41.539692 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78fb9a4-8ba3-4465-8b3c-228205660a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:41.541018 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78fb9a4-8ba3-4465-8b3c-228205660a38 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:41.541499 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78fb9a4-8ba3-4465-8b3c-228205660a38 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:41.542208 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78fb9a4-8ba3-4465-8b3c-228205660a38 None admin] PATH_INFO: `/v3/users/5e3ac3a673394ce79418e671768a7458` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:41.544574 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78fb9a4-8ba3-4465-8b3c-228205660a38 None admin] RBAC: Authorizing `identity:get_user(user_id=5e3ac3a673394ce79418e671768a7458)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:41.546401 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78fb9a4-8ba3-4465-8b3c-228205660a38 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:41.553619 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 115/228] 10.222.0.55 () {56 vars in 1177 bytes} [Wed Aug 24 08:33:41 2022] GET /identity/v3/users/5e3ac3a673394ce79418e671768a7458 => generated 283 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:41.554736 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78fb9a4-8ba3-4465-8b3c-228205660a38 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:41.579331 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58914467-3ae5-48f9-afe3-b3fc4d77cb1e None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:41.616807 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98413cd-4d07-4faf-aa55-6c42f1280c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:41.617374 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98413cd-4d07-4faf-aa55-6c42f1280c8f None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:41.618011 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98413cd-4d07-4faf-aa55-6c42f1280c8f None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:41.618011 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98413cd-4d07-4faf-aa55-6c42f1280c8f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:41.619304 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98413cd-4d07-4faf-aa55-6c42f1280c8f None admin] RBAC: Adding query filter params (role.id=b674fd0a592e4e0cada811bba5ceea4d, scope.project.id=61247a1049b846b79052f42b0d525259, user.id=10ab6b64fbda471a8f7756070346fe3b) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:33:41.619615 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98413cd-4d07-4faf-aa55-6c42f1280c8f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:41.621137 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98413cd-4d07-4faf-aa55-6c42f1280c8f None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:41.628398 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 114/229] 10.222.0.55 () {56 vars in 1374 bytes} [Wed Aug 24 08:33:41 2022] GET /identity/v3/role_assignments?user.id=10ab6b64fbda471a8f7756070346fe3b&scope.project.id=61247a1049b846b79052f42b0d525259&role.id=b674fd0a592e4e0cada811bba5ceea4d => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:41.629247 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98413cd-4d07-4faf-aa55-6c42f1280c8f None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:41.669712 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78fb9a4-8ba3-4465-8b3c-228205660a38 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:41.689307 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:41.707989 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c440468f-f461-492d-b804-b4cb963e299f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:41.709052 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c440468f-f461-492d-b804-b4cb963e299f None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:41.709535 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c440468f-f461-492d-b804-b4cb963e299f None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:41.710006 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c440468f-f461-492d-b804-b4cb963e299f None admin] PATH_INFO: `/v3/users/5e3ac3a673394ce79418e671768a7458` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:41.711659 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c440468f-f461-492d-b804-b4cb963e299f None admin] RBAC: Authorizing `identity:get_user(user_id=5e3ac3a673394ce79418e671768a7458)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:41.713526 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c440468f-f461-492d-b804-b4cb963e299f None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:41.715828 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 116/230] 10.222.0.55 () {56 vars in 1177 bytes} [Wed Aug 24 08:33:41 2022] GET /identity/v3/users/5e3ac3a673394ce79418e671768a7458 => generated 283 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:41.722231 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c440468f-f461-492d-b804-b4cb963e299f None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:41.727699 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:41.757853 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98413cd-4d07-4faf-aa55-6c42f1280c8f None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:41.799396 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67306add-203a-4b71-a0fc-e4eac80d9bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:41.800392 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67306add-203a-4b71-a0fc-e4eac80d9bad None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:41.800895 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67306add-203a-4b71-a0fc-e4eac80d9bad None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:41.801504 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67306add-203a-4b71-a0fc-e4eac80d9bad None admin] PATH_INFO: `/v3/projects/61247a1049b846b79052f42b0d525259` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:41.802980 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67306add-203a-4b71-a0fc-e4eac80d9bad None admin] RBAC: Authorizing `identity:get_project(project_id=61247a1049b846b79052f42b0d525259)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:41.804670 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67306add-203a-4b71-a0fc-e4eac80d9bad None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:41.807667 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 115/231] 10.222.0.55 () {56 vars in 1186 bytes} [Wed Aug 24 08:33:41 2022] GET /identity/v3/projects/61247a1049b846b79052f42b0d525259 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:41.822978 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67306add-203a-4b71-a0fc-e4eac80d9bad None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:41.851426 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c440468f-f461-492d-b804-b4cb963e299f None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:41.911127 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeac1cb-f3ec-441f-9a2c-c594e96df2b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:41.911729 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeac1cb-f3ec-441f-9a2c-c594e96df2b5 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:41.912035 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeac1cb-f3ec-441f-9a2c-c594e96df2b5 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:41.920624 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeac1cb-f3ec-441f-9a2c-c594e96df2b5 None admin] PATH_INFO: `/v3/projects/61247a1049b846b79052f42b0d525259` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:41.920624 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eeac1cb-f3ec-441f-9a2c-c594e96df2b5 None admin] RBAC: Authorizing `identity:get_project(project_id=61247a1049b846b79052f42b0d525259)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:41.920624 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eeac1cb-f3ec-441f-9a2c-c594e96df2b5 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:41.920624 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 117/232] 10.222.0.55 () {56 vars in 1186 bytes} [Wed Aug 24 08:33:41 2022] GET /identity/v3/projects/61247a1049b846b79052f42b0d525259 => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:41.924461 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeac1cb-f3ec-441f-9a2c-c594e96df2b5 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:41.975039 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67306add-203a-4b71-a0fc-e4eac80d9bad None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:41.995849 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:42.047896 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f80bbd-457e-4e25-bddc-6134532ee78c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:42.048802 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f80bbd-457e-4e25-bddc-6134532ee78c None admin] REQUEST_METHOD: `PUT` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:42.049170 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f80bbd-457e-4e25-bddc-6134532ee78c None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:42.049540 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f80bbd-457e-4e25-bddc-6134532ee78c None admin] PATH_INFO: `/v3/projects/61247a1049b846b79052f42b0d525259/users/5e3ac3a673394ce79418e671768a7458/roles/b870cd1acaa744e1ae1d58b9ac283821` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:42.051628 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f80bbd-457e-4e25-bddc-6134532ee78c None admin] RBAC: Authorizing `identity:create_grant(project_id=61247a1049b846b79052f42b0d525259, user_id=5e3ac3a673394ce79418e671768a7458, role_id=b870cd1acaa744e1ae1d58b9ac283821)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:42.053215 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f80bbd-457e-4e25-bddc-6134532ee78c None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:42.054846 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeac1cb-f3ec-441f-9a2c-c594e96df2b5 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:42.093506 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b52a4d-6a9c-4bbc-801b-59958e254e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:42.094415 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b52a4d-6a9c-4bbc-801b-59958e254e63 None admin] REQUEST_METHOD: `PUT` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:42.094914 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b52a4d-6a9c-4bbc-801b-59958e254e63 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:42.095338 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b52a4d-6a9c-4bbc-801b-59958e254e63 None admin] PATH_INFO: `/v3/projects/61247a1049b846b79052f42b0d525259/users/5e3ac3a673394ce79418e671768a7458/roles/8b5ad96041894155b0138a676ead607e` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:42.097714 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b52a4d-6a9c-4bbc-801b-59958e254e63 None admin] RBAC: Authorizing `identity:create_grant(project_id=61247a1049b846b79052f42b0d525259, user_id=5e3ac3a673394ce79418e671768a7458, role_id=8b5ad96041894155b0138a676ead607e)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:42.099310 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b52a4d-6a9c-4bbc-801b-59958e254e63 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:42.138602 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 116/233] 10.222.0.55 () {58 vars in 1439 bytes} [Wed Aug 24 08:33:41 2022] PUT /identity/v3/projects/61247a1049b846b79052f42b0d525259/users/5e3ac3a673394ce79418e671768a7458/roles/b870cd1acaa744e1ae1d58b9ac283821 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:33:42.195360 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 118/234] 10.222.0.55 () {58 vars in 1439 bytes} [Wed Aug 24 08:33:41 2022] PUT /identity/v3/projects/61247a1049b846b79052f42b0d525259/users/5e3ac3a673394ce79418e671768a7458/roles/8b5ad96041894155b0138a676ead607e => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:33:42.977476 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8357eaa6-7a22-40ea-b4fc-2f89d8cb7817 None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:42.978006 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8357eaa6-7a22-40ea-b4fc-2f89d8cb7817 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:42.978006 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8357eaa6-7a22-40ea-b4fc-2f89d8cb7817 None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:42.979842 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 117/235] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:33:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:33:42.998947 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b4a296-c0be-47fb-ba9e-19993cb0d9ae None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:42.999459 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b4a296-c0be-47fb-ba9e-19993cb0d9ae None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:42.999533 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b4a296-c0be-47fb-ba9e-19993cb0d9ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:43.021548 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-79b4a296-c0be-47fb-ba9e-19993cb0d9ae None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:33:43.022818 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-79b4a296-c0be-47fb-ba9e-19993cb0d9ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:43.081680 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3d0824-3188-4b80-8054-f9381cbb468f None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:43.081997 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3d0824-3188-4b80-8054-f9381cbb468f None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:43.082230 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3d0824-3188-4b80-8054-f9381cbb468f None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:43.083892 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 118/236] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:33:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:33:43.090907 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67b6b28-b2ea-4743-8220-d13c752d2e7b None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:43.091184 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67b6b28-b2ea-4743-8220-d13c752d2e7b None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:43.091468 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67b6b28-b2ea-4743-8220-d13c752d2e7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:43.100087 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:43.124609 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 119/237] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:33:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:33:43.129817 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-e67b6b28-b2ea-4743-8220-d13c752d2e7b None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:33:43.131825 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-e67b6b28-b2ea-4743-8220-d13c752d2e7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:43.232761 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 119/238] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:33:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 08:33:43.513809 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b4a296-c0be-47fb-ba9e-19993cb0d9ae None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:43.564312 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:43.592197 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67b6b28-b2ea-4743-8220-d13c752d2e7b None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:43.666993 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b4a296-c0be-47fb-ba9e-19993cb0d9ae None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:43.743498 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1e6b76-b17d-4ab5-90aa-e0dcde5cd832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:43.744757 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1e6b76-b17d-4ab5-90aa-e0dcde5cd832 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:43.745391 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1e6b76-b17d-4ab5-90aa-e0dcde5cd832 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:43.746011 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1e6b76-b17d-4ab5-90aa-e0dcde5cd832 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:43.748516 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1e6b76-b17d-4ab5-90aa-e0dcde5cd832 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:43.750274 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1e6b76-b17d-4ab5-90aa-e0dcde5cd832 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:43.753225 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 120/239] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:33:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:43.759508 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1e6b76-b17d-4ab5-90aa-e0dcde5cd832 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:43.815823 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67b6b28-b2ea-4743-8220-d13c752d2e7b None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:43.889746 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50dd634-6f3c-42c4-a218-a7bdb6d8f0f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:43.890429 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50dd634-6f3c-42c4-a218-a7bdb6d8f0f3 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:43.890746 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50dd634-6f3c-42c4-a218-a7bdb6d8f0f3 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:43.891020 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50dd634-6f3c-42c4-a218-a7bdb6d8f0f3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:43.903730 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:43.913766 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50dd634-6f3c-42c4-a218-a7bdb6d8f0f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:33:43.919316 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-c50dd634-6f3c-42c4-a218-a7bdb6d8f0f3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 08:33:43.919316 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:33:43.919316 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:33:43.919316 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:33:43.919316 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:33:43.919316 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:33:43.919316 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:33:43.919316 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:33:43.919316 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:33:43.919316 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:33:43.919316 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:33:43.919316 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:33:43.919316 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:33:43.919316 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:33:43.919316 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:33:43.919316 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:33:43.920223 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:33:43.920223 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:33:43.920223 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:33:43.920223 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:33:43.920223 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:33:43.920223 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:33:43.920223 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:33:43.920223 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:33:43.920223 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:33:43.920223 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:33:43.920223 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:33:43.920223 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:33:43.920223 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:33:43.920223 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:33:43.920223 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:33:43.920223 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:33:43.921070 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:33:43.921070 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:33:43.921070 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:33:43.921070 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:33:43.921070 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:33:43.921070 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:33:43.921070 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:33:43.921070 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:33:43.921070 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:33:43.921070 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:33:43.921070 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 08:33:43.921070 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:33:43.921070 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 120/240] 10.222.0.55 () {56 vars in 1099 bytes} [Wed Aug 24 08:33:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:33:43.928576 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0175596-d23d-4f83-a819-869a86c707e7 None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:43.928576 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0175596-d23d-4f83-a819-869a86c707e7 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:43.928576 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0175596-d23d-4f83-a819-869a86c707e7 None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:43.928576 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 121/241] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:33:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:33:43.930084 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0175596-d23d-4f83-a819-869a86c707e7 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:43.950632 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1e6b76-b17d-4ab5-90aa-e0dcde5cd832 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:43.966354 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:44.000564 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf164ed-3707-436e-96ed-a5115180c529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:44.002429 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf164ed-3707-436e-96ed-a5115180c529 None admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:44.002827 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf164ed-3707-436e-96ed-a5115180c529 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:44.003644 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf164ed-3707-436e-96ed-a5115180c529 None admin] PATH_INFO: `/v3/users` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:44.004836 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf164ed-3707-436e-96ed-a5115180c529 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:44.006728 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf164ed-3707-436e-96ed-a5115180c529 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:44.053815 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0175596-d23d-4f83-a819-869a86c707e7 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:44.093566 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb908d42-a9a6-4662-a89d-a4a369572326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:44.094088 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb908d42-a9a6-4662-a89d-a4a369572326 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:44.094365 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb908d42-a9a6-4662-a89d-a4a369572326 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:44.094634 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb908d42-a9a6-4662-a89d-a4a369572326 None admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:44.095299 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb908d42-a9a6-4662-a89d-a4a369572326 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:33:44.095598 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb908d42-a9a6-4662-a89d-a4a369572326 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:44.097126 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb908d42-a9a6-4662-a89d-a4a369572326 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:44.108921 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 121/242] 10.222.0.55 () {60 vars in 1132 bytes} [Wed Aug 24 08:33:43 2022] POST /identity/v3/users => generated 293 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:33:44.116811 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708e1e95-64b0-47ae-b2bc-15b5fd8c3aba None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:44.116811 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708e1e95-64b0-47ae-b2bc-15b5fd8c3aba None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:44.116811 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708e1e95-64b0-47ae-b2bc-15b5fd8c3aba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:44.117155 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 122/243] 10.222.0.55 () {56 vars in 1101 bytes} [Wed Aug 24 08:33:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:44.117901 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb908d42-a9a6-4662-a89d-a4a369572326 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:44.149814 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-708e1e95-64b0-47ae-b2bc-15b5fd8c3aba None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:33:44.151972 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-708e1e95-64b0-47ae-b2bc-15b5fd8c3aba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:44.248976 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:44.309880 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 122/244] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:33:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:33:44.324460 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb908d42-a9a6-4662-a89d-a4a369572326 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:44.361014 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18484b34-8fad-4ad5-bc76-2127519c3729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:44.361917 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18484b34-8fad-4ad5-bc76-2127519c3729 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:44.362346 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18484b34-8fad-4ad5-bc76-2127519c3729 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:44.362764 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18484b34-8fad-4ad5-bc76-2127519c3729 None admin] PATH_INFO: `/v3/users/c63928f678444929bce84f3d0030dc8e` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:44.379612 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18484b34-8fad-4ad5-bc76-2127519c3729 None admin] RBAC: Authorizing `identity:get_user(user_id=c63928f678444929bce84f3d0030dc8e)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:44.381394 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18484b34-8fad-4ad5-bc76-2127519c3729 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:44.383616 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 123/245] 10.222.0.55 () {56 vars in 1177 bytes} [Wed Aug 24 08:33:44 2022] GET /identity/v3/users/c63928f678444929bce84f3d0030dc8e => generated 297 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:44.389580 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708e1e95-64b0-47ae-b2bc-15b5fd8c3aba None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:44.514069 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708e1e95-64b0-47ae-b2bc-15b5fd8c3aba None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:44.515948 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:44.554656 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48244cb5-b259-4046-b4f2-21f0a683060d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:44.555284 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48244cb5-b259-4046-b4f2-21f0a683060d None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:44.555546 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff2cb19-d3a6-4c33-a5cc-95c895f9ccda None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:44.555650 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48244cb5-b259-4046-b4f2-21f0a683060d None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:44.555803 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48244cb5-b259-4046-b4f2-21f0a683060d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:44.555915 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff2cb19-d3a6-4c33-a5cc-95c895f9ccda None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:44.556115 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff2cb19-d3a6-4c33-a5cc-95c895f9ccda None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:44.558257 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 124/246] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:33:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:33:44.565633 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e770a6-9539-4fa4-a8f5-cc19aa3d8c7d None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:44.566327 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e770a6-9539-4fa4-a8f5-cc19aa3d8c7d None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:44.566827 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e770a6-9539-4fa4-a8f5-cc19aa3d8c7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:44.594099 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-f9e770a6-9539-4fa4-a8f5-cc19aa3d8c7d None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:33:44.595942 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-f9e770a6-9539-4fa4-a8f5-cc19aa3d8c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:44.658554 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48244cb5-b259-4046-b4f2-21f0a683060d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:44.659977 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48244cb5-b259-4046-b4f2-21f0a683060d None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:44.698916 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 125/247] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:33:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 08:33:44.699941 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e770a6-9539-4fa4-a8f5-cc19aa3d8c7d None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:44.762274 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:44.782723 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:44.791417 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 123/248] 10.222.0.55 () {58 vars in 1282 bytes} [Wed Aug 24 08:33:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:33:44.793227 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efc69ab-85c9-4d13-9be0-76b45b6913ef None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:44.793609 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efc69ab-85c9-4d13-9be0-76b45b6913ef None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:44.793958 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efc69ab-85c9-4d13-9be0-76b45b6913ef None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:44.796423 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 124/249] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:33:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:33:44.801528 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305616d8-9514-4d31-81c2-3456f4d71606 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:44.801992 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305616d8-9514-4d31-81c2-3456f4d71606 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:44.802345 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305616d8-9514-4d31-81c2-3456f4d71606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:44.819405 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e770a6-9539-4fa4-a8f5-cc19aa3d8c7d None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:44.827089 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-305616d8-9514-4d31-81c2-3456f4d71606 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:33:44.828315 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-305616d8-9514-4d31-81c2-3456f4d71606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:44.862351 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d208a43-a7f5-445e-b676-aa7ccfe0f314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:44.863002 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d208a43-a7f5-445e-b676-aa7ccfe0f314 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:44.863286 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d208a43-a7f5-445e-b676-aa7ccfe0f314 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:44.863550 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d208a43-a7f5-445e-b676-aa7ccfe0f314 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:44.876404 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d208a43-a7f5-445e-b676-aa7ccfe0f314 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:33:44.881460 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-4d208a43-a7f5-445e-b676-aa7ccfe0f314 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:33:44.881460 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:33:44.881460 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:33:44.881460 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:33:44.881460 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:33:44.881460 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:33:44.881460 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:33:44.881460 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:33:44.881460 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:33:44.881460 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:33:44.881460 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:33:44.881460 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:33:44.881460 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:33:44.881460 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:33:44.881460 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:33:44.881460 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:33:44.882441 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:33:44.882441 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:33:44.882441 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:33:44.882441 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:33:44.882441 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:33:44.882441 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:33:44.882441 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:33:44.882441 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:33:44.882441 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:33:44.882441 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:33:44.882441 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:33:44.882441 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:33:44.882441 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:33:44.882441 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:33:44.882441 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:33:44.882441 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:33:44.883210 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:33:44.883210 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:33:44.883210 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:33:44.883210 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:33:44.883210 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:33:44.883210 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:33:44.883210 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:33:44.883210 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:33:44.883210 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:33:44.883210 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:33:44.883210 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:33:44.883210 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:33:44.883210 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 126/250] 10.222.0.55 () {56 vars in 1099 bytes} [Wed Aug 24 08:33:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:33:44.883855 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d208a43-a7f5-445e-b676-aa7ccfe0f314 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:44.925115 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 125/251] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:33:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:33:44.926185 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305616d8-9514-4d31-81c2-3456f4d71606 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:45.019967 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d208a43-a7f5-445e-b676-aa7ccfe0f314 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:45.024067 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:45.056149 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305616d8-9514-4d31-81c2-3456f4d71606 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:45.073389 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b4fb1e-f7c0-4f5e-b6c7-3f12f9ad8a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:45.074088 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b4fb1e-f7c0-4f5e-b6c7-3f12f9ad8a3c None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:45.074370 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b4fb1e-f7c0-4f5e-b6c7-3f12f9ad8a3c None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:45.074638 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b4fb1e-f7c0-4f5e-b6c7-3f12f9ad8a3c None admin] PATH_INFO: `/v3/projects/61247a1049b846b79052f42b0d525259` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:45.076343 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b4fb1e-f7c0-4f5e-b6c7-3f12f9ad8a3c None admin] RBAC: Authorizing `identity:get_project(project_id=61247a1049b846b79052f42b0d525259)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:45.078110 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b4fb1e-f7c0-4f5e-b6c7-3f12f9ad8a3c None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:45.080802 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 127/252] 10.222.0.55 () {56 vars in 1186 bytes} [Wed Aug 24 08:33:44 2022] GET /identity/v3/projects/61247a1049b846b79052f42b0d525259 => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:45.081700 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b4fb1e-f7c0-4f5e-b6c7-3f12f9ad8a3c None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:45.090708 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:45.108578 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b88be49-a8ef-43b5-bb6f-cfd340b07e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:45.108578 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b88be49-a8ef-43b5-bb6f-cfd340b07e07 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:45.108578 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b88be49-a8ef-43b5-bb6f-cfd340b07e07 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:45.108578 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b88be49-a8ef-43b5-bb6f-cfd340b07e07 None admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:45.108947 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b88be49-a8ef-43b5-bb6f-cfd340b07e07 None admin] RBAC: Adding query filter params (name=member) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:33:45.108947 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b88be49-a8ef-43b5-bb6f-cfd340b07e07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:45.108947 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b88be49-a8ef-43b5-bb6f-cfd340b07e07 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:45.114869 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 126/253] 10.222.0.55 () {56 vars in 1101 bytes} [Wed Aug 24 08:33:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:45.115655 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b88be49-a8ef-43b5-bb6f-cfd340b07e07 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:45.235571 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b88be49-a8ef-43b5-bb6f-cfd340b07e07 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:45.274288 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac50594f-8903-464f-86bd-cb6ef09f8da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:45.275114 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac50594f-8903-464f-86bd-cb6ef09f8da8 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:45.275482 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac50594f-8903-464f-86bd-cb6ef09f8da8 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:45.275823 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac50594f-8903-464f-86bd-cb6ef09f8da8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:45.280563 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac50594f-8903-464f-86bd-cb6ef09f8da8 None admin] RBAC: Adding query filter params (role.id=e308031ba9a84b79bcdf501455ebe0d7, scope.project.id=61247a1049b846b79052f42b0d525259, user.id=c63928f678444929bce84f3d0030dc8e) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:33:45.280563 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac50594f-8903-464f-86bd-cb6ef09f8da8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:45.280563 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac50594f-8903-464f-86bd-cb6ef09f8da8 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:45.284795 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 127/254] 10.222.0.55 () {56 vars in 1374 bytes} [Wed Aug 24 08:33:45 2022] GET /identity/v3/role_assignments?user.id=c63928f678444929bce84f3d0030dc8e&scope.project.id=61247a1049b846b79052f42b0d525259&role.id=e308031ba9a84b79bcdf501455ebe0d7 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:45.285800 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac50594f-8903-464f-86bd-cb6ef09f8da8 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:45.313863 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b4fb1e-f7c0-4f5e-b6c7-3f12f9ad8a3c None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:45.348947 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:45.390457 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4bec3e-f916-4b17-8814-36a89ae24dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:45.391120 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4bec3e-f916-4b17-8814-36a89ae24dee None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:45.391402 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4bec3e-f916-4b17-8814-36a89ae24dee None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:45.391677 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4bec3e-f916-4b17-8814-36a89ae24dee None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:45.408269 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac50594f-8903-464f-86bd-cb6ef09f8da8 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:45.419925 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4bec3e-f916-4b17-8814-36a89ae24dee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:33:45.424897 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-aa4bec3e-f916-4b17-8814-36a89ae24dee None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:33:45.424897 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:33:45.424897 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:33:45.424897 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:33:45.424897 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:33:45.424897 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:33:45.424897 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:33:45.424897 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:33:45.424897 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:33:45.424897 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:33:45.424897 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:33:45.424897 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:33:45.424897 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:33:45.424897 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:33:45.424897 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:33:45.424897 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:33:45.425849 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:33:45.425849 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:33:45.425849 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:33:45.425849 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:33:45.425849 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:33:45.425849 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:33:45.425849 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:33:45.425849 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:33:45.425849 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:33:45.425849 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:33:45.425849 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:33:45.425849 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:33:45.425849 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:33:45.425849 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:33:45.425849 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:33:45.425849 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:33:45.427063 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:33:45.427063 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:33:45.427063 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:33:45.427063 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:33:45.427063 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:33:45.427063 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:33:45.427063 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:33:45.427063 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:33:45.427063 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:33:45.427063 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:33:45.427063 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:33:45.427063 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:33:45.427063 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 128/255] 10.222.0.55 () {56 vars in 1096 bytes} [Wed Aug 24 08:33:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:33:45.427063 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4bec3e-f916-4b17-8814-36a89ae24dee None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:45.451439 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57761db1-2832-4b78-b056-fe160717b213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:45.452133 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57761db1-2832-4b78-b056-fe160717b213 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:45.452618 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57761db1-2832-4b78-b056-fe160717b213 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:45.453195 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57761db1-2832-4b78-b056-fe160717b213 None admin] PATH_INFO: `/v3/users/10ab6b64fbda471a8f7756070346fe3b` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:45.454572 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57761db1-2832-4b78-b056-fe160717b213 None admin] RBAC: Authorizing `identity:get_user(user_id=10ab6b64fbda471a8f7756070346fe3b)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:45.456264 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57761db1-2832-4b78-b056-fe160717b213 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:45.458427 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 128/256] 10.222.0.55 () {56 vars in 1177 bytes} [Wed Aug 24 08:33:45 2022] GET /identity/v3/users/10ab6b64fbda471a8f7756070346fe3b => generated 297 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:45.459534 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57761db1-2832-4b78-b056-fe160717b213 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:45.509372 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:45.585742 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57761db1-2832-4b78-b056-fe160717b213 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:45.593584 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4bec3e-f916-4b17-8814-36a89ae24dee None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:45.608916 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:45.625393 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0396d10-9f0c-42ac-89e0-7e8b294483cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:45.626068 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0396d10-9f0c-42ac-89e0-7e8b294483cd None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:45.626279 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0396d10-9f0c-42ac-89e0-7e8b294483cd None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:45.626556 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0396d10-9f0c-42ac-89e0-7e8b294483cd None admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:45.627230 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0396d10-9f0c-42ac-89e0-7e8b294483cd None admin] RBAC: Adding query filter params (name=admin) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:33:45.627535 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0396d10-9f0c-42ac-89e0-7e8b294483cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:45.629169 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0396d10-9f0c-42ac-89e0-7e8b294483cd None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:45.638042 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ba2952-c561-4891-872d-920335d52f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:45.639027 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ba2952-c561-4891-872d-920335d52f1a None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:45.639506 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ba2952-c561-4891-872d-920335d52f1a None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:45.639817 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 129/257] 10.222.0.55 () {56 vars in 1099 bytes} [Wed Aug 24 08:33:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:45.640245 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ba2952-c561-4891-872d-920335d52f1a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:45.640838 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0396d10-9f0c-42ac-89e0-7e8b294483cd None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:45.654445 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ba2952-c561-4891-872d-920335d52f1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:33:45.660054 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-74ba2952-c561-4891-872d-920335d52f1a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 08:33:45.660054 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:33:45.660054 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:33:45.660054 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:33:45.660054 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:33:45.660054 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:33:45.660054 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:33:45.660054 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:33:45.660054 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:33:45.660054 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:33:45.660054 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:33:45.660054 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:33:45.660054 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:33:45.660054 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:33:45.660054 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:33:45.660054 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:33:45.661049 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:33:45.661049 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:33:45.661049 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:33:45.661049 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:33:45.661049 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:33:45.661049 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:33:45.661049 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:33:45.661049 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:33:45.661049 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:33:45.661049 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:33:45.661049 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:33:45.661049 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:33:45.661049 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:33:45.661049 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:33:45.661049 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:33:45.661049 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:33:45.661860 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:33:45.661860 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:33:45.661860 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:33:45.661860 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:33:45.661860 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:33:45.661860 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:33:45.661860 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:33:45.661860 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:33:45.661860 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:33:45.661860 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:33:45.661860 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 08:33:45.661860 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:33:45.664214 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 129/258] 10.222.0.55 () {56 vars in 1114 bytes} [Wed Aug 24 08:33:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:33:45.666249 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ba2952-c561-4891-872d-920335d52f1a None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:45.755288 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0396d10-9f0c-42ac-89e0-7e8b294483cd None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:45.786490 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:45.799567 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73685846-e07b-4672-b3fe-2a323d81c56a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:45.801058 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73685846-e07b-4672-b3fe-2a323d81c56a None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:45.801335 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73685846-e07b-4672-b3fe-2a323d81c56a None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:45.801604 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73685846-e07b-4672-b3fe-2a323d81c56a None admin] PATH_INFO: `/v3/projects/61247a1049b846b79052f42b0d525259` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:45.804376 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73685846-e07b-4672-b3fe-2a323d81c56a None admin] RBAC: Authorizing `identity:get_project(project_id=61247a1049b846b79052f42b0d525259)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:45.805863 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73685846-e07b-4672-b3fe-2a323d81c56a None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:45.807865 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ba2952-c561-4891-872d-920335d52f1a None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:45.810000 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 130/259] 10.222.0.55 () {56 vars in 1186 bytes} [Wed Aug 24 08:33:45 2022] GET /identity/v3/projects/61247a1049b846b79052f42b0d525259 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:45.810599 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73685846-e07b-4672-b3fe-2a323d81c56a None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:45.882079 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c71283-3cc8-4ae6-ae2f-365b9960e676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:45.882732 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c71283-3cc8-4ae6-ae2f-365b9960e676 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:45.883068 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c71283-3cc8-4ae6-ae2f-365b9960e676 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:45.883339 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c71283-3cc8-4ae6-ae2f-365b9960e676 None admin] PATH_INFO: `/v3/users/5e3ac3a673394ce79418e671768a7458` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:45.885573 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c71283-3cc8-4ae6-ae2f-365b9960e676 None admin] RBAC: Authorizing `identity:get_user(user_id=5e3ac3a673394ce79418e671768a7458)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:45.887111 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c71283-3cc8-4ae6-ae2f-365b9960e676 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:45.893922 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 130/260] 10.222.0.55 () {56 vars in 1177 bytes} [Wed Aug 24 08:33:45 2022] GET /identity/v3/users/5e3ac3a673394ce79418e671768a7458 => generated 283 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:45.893922 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c71283-3cc8-4ae6-ae2f-365b9960e676 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:45.928922 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:46.031652 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73685846-e07b-4672-b3fe-2a323d81c56a None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:46.122955 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c71283-3cc8-4ae6-ae2f-365b9960e676 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:46.137761 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91299e9b-f3c1-4e7f-b085-b5a688b67d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:46.138725 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91299e9b-f3c1-4e7f-b085-b5a688b67d8c None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:46.139235 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91299e9b-f3c1-4e7f-b085-b5a688b67d8c None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:46.139735 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91299e9b-f3c1-4e7f-b085-b5a688b67d8c None admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:46.140955 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91299e9b-f3c1-4e7f-b085-b5a688b67d8c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:33:46.141515 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91299e9b-f3c1-4e7f-b085-b5a688b67d8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:46.144190 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91299e9b-f3c1-4e7f-b085-b5a688b67d8c None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:46.167362 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 131/261] 10.222.0.55 () {56 vars in 1111 bytes} [Wed Aug 24 08:33:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:46.168201 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91299e9b-f3c1-4e7f-b085-b5a688b67d8c None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:46.222345 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d10e5a9-2b81-45be-887c-ca2ebdc8fe38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:46.223164 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d10e5a9-2b81-45be-887c-ca2ebdc8fe38 None admin] REQUEST_METHOD: `PUT` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:46.223448 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d10e5a9-2b81-45be-887c-ca2ebdc8fe38 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:46.223726 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d10e5a9-2b81-45be-887c-ca2ebdc8fe38 None admin] PATH_INFO: `/v3/projects/61247a1049b846b79052f42b0d525259/users/10ab6b64fbda471a8f7756070346fe3b/roles/b674fd0a592e4e0cada811bba5ceea4d` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:46.226001 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d10e5a9-2b81-45be-887c-ca2ebdc8fe38 None admin] RBAC: Authorizing `identity:create_grant(project_id=61247a1049b846b79052f42b0d525259, user_id=10ab6b64fbda471a8f7756070346fe3b, role_id=b674fd0a592e4e0cada811bba5ceea4d)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:46.227508 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d10e5a9-2b81-45be-887c-ca2ebdc8fe38 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:46.302616 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 131/262] 10.222.0.55 () {58 vars in 1439 bytes} [Wed Aug 24 08:33:45 2022] PUT /identity/v3/projects/61247a1049b846b79052f42b0d525259/users/10ab6b64fbda471a8f7756070346fe3b/roles/b674fd0a592e4e0cada811bba5ceea4d => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:33:46.303146 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d10e5a9-2b81-45be-887c-ca2ebdc8fe38 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:46.357026 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:46.367456 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91299e9b-f3c1-4e7f-b085-b5a688b67d8c None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:46.404452 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:46.472250 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c3c9ac-0314-4fe0-887f-f2f6ddc5b09d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:46.472919 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c3c9ac-0314-4fe0-887f-f2f6ddc5b09d None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:46.473197 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c3c9ac-0314-4fe0-887f-f2f6ddc5b09d None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:46.473468 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c3c9ac-0314-4fe0-887f-f2f6ddc5b09d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:46.566711 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d10e5a9-2b81-45be-887c-ca2ebdc8fe38 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:46.653083 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8362ce7-8a2a-4662-8074-e417a1ad5e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:46.653700 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8362ce7-8a2a-4662-8074-e417a1ad5e30 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:46.653963 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8362ce7-8a2a-4662-8074-e417a1ad5e30 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:46.654239 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8362ce7-8a2a-4662-8074-e417a1ad5e30 None admin] PATH_INFO: `/v3/users/5e3ac3a673394ce79418e671768a7458` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:46.656526 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8362ce7-8a2a-4662-8074-e417a1ad5e30 None admin] RBAC: Authorizing `identity:get_user(user_id=5e3ac3a673394ce79418e671768a7458)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:46.658034 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8362ce7-8a2a-4662-8074-e417a1ad5e30 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:46.660248 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 132/263] 10.222.0.55 () {56 vars in 1177 bytes} [Wed Aug 24 08:33:46 2022] GET /identity/v3/users/5e3ac3a673394ce79418e671768a7458 => generated 283 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:46.660815 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c3c9ac-0314-4fe0-887f-f2f6ddc5b09d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:46.662428 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c3c9ac-0314-4fe0-887f-f2f6ddc5b09d None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:46.667330 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8362ce7-8a2a-4662-8074-e417a1ad5e30 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:46.864680 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:46.895983 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8362ce7-8a2a-4662-8074-e417a1ad5e30 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:46.900207 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 132/264] 10.222.0.55 () {58 vars in 1282 bytes} [Wed Aug 24 08:33:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 732 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:33:46.912411 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c3c9ac-0314-4fe0-887f-f2f6ddc5b09d None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:46.940913 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:46.983885 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fe3330-e0bc-4b40-9197-db09ef16789f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:46.984532 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fe3330-e0bc-4b40-9197-db09ef16789f None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:46.984854 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fe3330-e0bc-4b40-9197-db09ef16789f None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:46.985107 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fe3330-e0bc-4b40-9197-db09ef16789f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:47.129723 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fe3330-e0bc-4b40-9197-db09ef16789f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:47.131143 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fe3330-e0bc-4b40-9197-db09ef16789f None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:47.160158 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c3c9ac-0314-4fe0-887f-f2f6ddc5b09d None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:47.237598 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e0ac59-4a31-4c9c-9605-2f272489eaa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:47.238258 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e0ac59-4a31-4c9c-9605-2f272489eaa3 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:47.238542 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e0ac59-4a31-4c9c-9605-2f272489eaa3 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:47.238819 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e0ac59-4a31-4c9c-9605-2f272489eaa3 None admin] PATH_INFO: `/v3/projects/61247a1049b846b79052f42b0d525259` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:47.246077 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e0ac59-4a31-4c9c-9605-2f272489eaa3 None admin] RBAC: Authorizing `identity:get_project(project_id=61247a1049b846b79052f42b0d525259)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:47.247318 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e0ac59-4a31-4c9c-9605-2f272489eaa3 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:47.249751 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 133/265] 10.222.0.55 () {56 vars in 1186 bytes} [Wed Aug 24 08:33:46 2022] GET /identity/v3/projects/61247a1049b846b79052f42b0d525259 => generated 302 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:47.251129 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a24055-767c-400f-9037-280b0a81cb8b None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:47.251418 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a24055-767c-400f-9037-280b0a81cb8b None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:47.251697 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a24055-767c-400f-9037-280b0a81cb8b None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:47.253178 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 134/266] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:33:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:33:47.254995 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a24055-767c-400f-9037-280b0a81cb8b None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:47.320766 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:47.343108 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 133/267] 10.222.0.55 () {58 vars in 1282 bytes} [Wed Aug 24 08:33:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 676 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:33:47.344804 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddc6848-ed2d-4d52-9711-45cce30bd764 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:47.345269 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddc6848-ed2d-4d52-9711-45cce30bd764 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:47.345688 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddc6848-ed2d-4d52-9711-45cce30bd764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:47.370815 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-8ddc6848-ed2d-4d52-9711-45cce30bd764 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:33:47.372266 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-8ddc6848-ed2d-4d52-9711-45cce30bd764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:47.383131 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:47.407398 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a24055-767c-400f-9037-280b0a81cb8b None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:47.452056 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b879fe89-9039-40b4-9ab4-cd26fd4f4310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:47.452720 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b879fe89-9039-40b4-9ab4-cd26fd4f4310 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:47.453002 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b879fe89-9039-40b4-9ab4-cd26fd4f4310 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:47.453320 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b879fe89-9039-40b4-9ab4-cd26fd4f4310 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:47.454346 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b879fe89-9039-40b4-9ab4-cd26fd4f4310 None admin] RBAC: Adding query filter params (role.id=b870cd1acaa744e1ae1d58b9ac283821, scope.project.id=61247a1049b846b79052f42b0d525259, user.id=5e3ac3a673394ce79418e671768a7458) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:33:47.454495 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b879fe89-9039-40b4-9ab4-cd26fd4f4310 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:47.455986 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b879fe89-9039-40b4-9ab4-cd26fd4f4310 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:47.463990 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 135/268] 10.222.0.55 () {56 vars in 1374 bytes} [Wed Aug 24 08:33:47 2022] GET /identity/v3/role_assignments?user.id=5e3ac3a673394ce79418e671768a7458&scope.project.id=61247a1049b846b79052f42b0d525259&role.id=b870cd1acaa744e1ae1d58b9ac283821 => generated 608 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:47.465517 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6716b7e9-be7a-4936-9509-9036c3a9ab0c None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:47.465803 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6716b7e9-be7a-4936-9509-9036c3a9ab0c None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:47.466084 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6716b7e9-be7a-4936-9509-9036c3a9ab0c None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:47.472135 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 136/269] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:33:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:33:47.472135 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6716b7e9-be7a-4936-9509-9036c3a9ab0c None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:47.484738 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 134/270] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:33:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:33:47.486537 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4661d4e4-9b5e-491e-be72-cf5dba2da2b2 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:47.486928 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4661d4e4-9b5e-491e-be72-cf5dba2da2b2 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:47.487327 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4661d4e4-9b5e-491e-be72-cf5dba2da2b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:47.526846 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-4661d4e4-9b5e-491e-be72-cf5dba2da2b2 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:33:47.528295 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-4661d4e4-9b5e-491e-be72-cf5dba2da2b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:47.613514 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6716b7e9-be7a-4936-9509-9036c3a9ab0c None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:47.660245 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f423cf-a880-4b20-b8e6-db8c207161c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:47.661291 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f423cf-a880-4b20-b8e6-db8c207161c8 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:47.661806 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f423cf-a880-4b20-b8e6-db8c207161c8 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:47.662283 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f423cf-a880-4b20-b8e6-db8c207161c8 None admin] PATH_INFO: `/v3/projects/61247a1049b846b79052f42b0d525259` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:47.663889 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f423cf-a880-4b20-b8e6-db8c207161c8 None admin] RBAC: Authorizing `identity:get_project(project_id=61247a1049b846b79052f42b0d525259)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:47.665732 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f423cf-a880-4b20-b8e6-db8c207161c8 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:47.672709 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 137/271] 10.222.0.55 () {56 vars in 1186 bytes} [Wed Aug 24 08:33:47 2022] GET /identity/v3/projects/61247a1049b846b79052f42b0d525259 => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:47.675641 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f423cf-a880-4b20-b8e6-db8c207161c8 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:47.696769 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:47.758242 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 135/272] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:33:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:33:47.804555 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4661d4e4-9b5e-491e-be72-cf5dba2da2b2 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:47.808491 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f423cf-a880-4b20-b8e6-db8c207161c8 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:47.847732 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1771d8ac-1ffe-4585-bb0d-2599ca23f44e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:47.848803 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1771d8ac-1ffe-4585-bb0d-2599ca23f44e None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:47.849289 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1771d8ac-1ffe-4585-bb0d-2599ca23f44e None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:47.849831 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1771d8ac-1ffe-4585-bb0d-2599ca23f44e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:47.850901 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1771d8ac-1ffe-4585-bb0d-2599ca23f44e None admin] RBAC: Adding query filter params (role.id=8b5ad96041894155b0138a676ead607e, scope.project.id=61247a1049b846b79052f42b0d525259, user.id=5e3ac3a673394ce79418e671768a7458) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:33:47.851477 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1771d8ac-1ffe-4585-bb0d-2599ca23f44e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:47.853191 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1771d8ac-1ffe-4585-bb0d-2599ca23f44e None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:47.863470 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 138/273] 10.222.0.55 () {56 vars in 1374 bytes} [Wed Aug 24 08:33:47 2022] GET /identity/v3/role_assignments?user.id=5e3ac3a673394ce79418e671768a7458&scope.project.id=61247a1049b846b79052f42b0d525259&role.id=8b5ad96041894155b0138a676ead607e => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:33:47.935709 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:48.015945 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4661d4e4-9b5e-491e-be72-cf5dba2da2b2 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:48.059309 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb537ca8-936b-46ca-b717-2d35448fa67d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:48.059963 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb537ca8-936b-46ca-b717-2d35448fa67d None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:48.060241 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb537ca8-936b-46ca-b717-2d35448fa67d None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:48.060512 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb537ca8-936b-46ca-b717-2d35448fa67d None admin] PATH_INFO: `/v3/domains/default` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:48.061697 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb537ca8-936b-46ca-b717-2d35448fa67d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:48.063122 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb537ca8-936b-46ca-b717-2d35448fa67d None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:48.065026 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 136/274] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:33:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:48.070539 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1771d8ac-1ffe-4585-bb0d-2599ca23f44e None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:48.117825 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb537ca8-936b-46ca-b717-2d35448fa67d None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:48.173373 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:48.208849 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1771d8ac-1ffe-4585-bb0d-2599ca23f44e None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:48.259359 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cea30cc-4571-4c65-bff1-6393abdb7a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:48.260485 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cea30cc-4571-4c65-bff1-6393abdb7a17 None admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:48.261326 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cea30cc-4571-4c65-bff1-6393abdb7a17 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:48.261741 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cea30cc-4571-4c65-bff1-6393abdb7a17 None admin] PATH_INFO: `/v3/users` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:48.262782 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cea30cc-4571-4c65-bff1-6393abdb7a17 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:48.265180 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cea30cc-4571-4c65-bff1-6393abdb7a17 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:48.270363 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb537ca8-936b-46ca-b717-2d35448fa67d None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:48.298121 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:48.318330 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a5ff17-c4c0-4a83-b112-80b88c3b5b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:48.319295 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a5ff17-c4c0-4a83-b112-80b88c3b5b7d None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:48.319574 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a5ff17-c4c0-4a83-b112-80b88c3b5b7d None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:48.319844 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a5ff17-c4c0-4a83-b112-80b88c3b5b7d None admin] PATH_INFO: `/v3/roles/member` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:48.334333 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a5ff17-c4c0-4a83-b112-80b88c3b5b7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:33:48.339833 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-08a5ff17-c4c0-4a83-b112-80b88c3b5b7d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:33:48.339833 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:33:48.339833 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:33:48.339833 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:33:48.339833 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:33:48.339833 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:33:48.339833 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:33:48.339833 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:33:48.339833 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:33:48.339833 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:33:48.339833 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:33:48.339833 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:33:48.339833 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:33:48.339833 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:33:48.339833 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:33:48.339833 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:33:48.340770 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:33:48.340770 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:33:48.340770 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:33:48.340770 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:33:48.340770 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:33:48.340770 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:33:48.340770 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:33:48.340770 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:33:48.340770 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:33:48.340770 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:33:48.340770 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:33:48.340770 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:33:48.340770 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:33:48.340770 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:33:48.340770 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:33:48.340770 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:33:48.342061 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:33:48.342061 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:33:48.342061 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:33:48.342061 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:33:48.342061 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:33:48.342061 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:33:48.342061 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:33:48.342061 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:33:48.342061 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:33:48.342061 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:33:48.342061 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:33:48.342061 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:33:48.342061 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 137/275] 10.222.0.55 () {56 vars in 1099 bytes} [Wed Aug 24 08:33:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:33:48.347134 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a5ff17-c4c0-4a83-b112-80b88c3b5b7d None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:48.386923 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 139/276] 10.222.0.55 () {60 vars in 1132 bytes} [Wed Aug 24 08:33:48 2022] POST /identity/v3/users => generated 293 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:33:48.464708 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7bfa4d-a490-4995-8f7a-15cdf62a42b3 None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:48.464985 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7bfa4d-a490-4995-8f7a-15cdf62a42b3 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:48.465195 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7bfa4d-a490-4995-8f7a-15cdf62a42b3 None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:48.466896 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 140/277] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:33:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:33:48.472638 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf0b292-0799-4e1a-8f48-2230d4508441 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:48.472996 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf0b292-0799-4e1a-8f48-2230d4508441 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:48.473280 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf0b292-0799-4e1a-8f48-2230d4508441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:48.474663 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a5ff17-c4c0-4a83-b112-80b88c3b5b7d None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:48.495263 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-faf0b292-0799-4e1a-8f48-2230d4508441 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:33:48.496512 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-faf0b292-0799-4e1a-8f48-2230d4508441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:48.523683 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c55607c-dcdf-4ff2-8264-5a47b7da386a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:48.524335 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c55607c-dcdf-4ff2-8264-5a47b7da386a None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:48.524644 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c55607c-dcdf-4ff2-8264-5a47b7da386a None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:48.525205 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c55607c-dcdf-4ff2-8264-5a47b7da386a None admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:48.526552 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c55607c-dcdf-4ff2-8264-5a47b7da386a None admin] RBAC: Adding query filter params (name=member) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:33:48.526869 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c55607c-dcdf-4ff2-8264-5a47b7da386a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:48.528646 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c55607c-dcdf-4ff2-8264-5a47b7da386a None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:48.542423 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 138/278] 10.222.0.55 () {56 vars in 1101 bytes} [Wed Aug 24 08:33:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:33:48.548998 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c55607c-dcdf-4ff2-8264-5a47b7da386a None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:48.584141 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 141/279] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:33:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:33:48.599027 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:48.667646 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c55607c-dcdf-4ff2-8264-5a47b7da386a None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:48.705228 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b81493-891b-4b36-aa81-c5d60a598451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:48.705877 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b81493-891b-4b36-aa81-c5d60a598451 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:48.706157 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b81493-891b-4b36-aa81-c5d60a598451 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:48.706430 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b81493-891b-4b36-aa81-c5d60a598451 None admin] PATH_INFO: `/v3/users/d108dc5b598d48b19e5232719ef33dc1` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:48.725432 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b81493-891b-4b36-aa81-c5d60a598451 None admin] RBAC: Authorizing `identity:get_user(user_id=d108dc5b598d48b19e5232719ef33dc1)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:48.726870 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b81493-891b-4b36-aa81-c5d60a598451 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:48.729610 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 139/280] 10.222.0.55 () {56 vars in 1177 bytes} [Wed Aug 24 08:33:48 2022] GET /identity/v3/users/d108dc5b598d48b19e5232719ef33dc1 => generated 293 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:33:48.735762 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf0b292-0799-4e1a-8f48-2230d4508441 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:48.807991 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:48.828512 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379fd98b-14a6-4140-9ee5-8616d10d7b85 None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:48.828973 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379fd98b-14a6-4140-9ee5-8616d10d7b85 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:48.829353 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379fd98b-14a6-4140-9ee5-8616d10d7b85 None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:48.831159 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 140/281] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:33:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:33:48.837178 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaa88c9-c4cd-405e-ac0f-dc29f4a78562 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:48.837592 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaa88c9-c4cd-405e-ac0f-dc29f4a78562 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:48.838502 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaa88c9-c4cd-405e-ac0f-dc29f4a78562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:48.860990 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf0b292-0799-4e1a-8f48-2230d4508441 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:48.864913 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-abaa88c9-c4cd-405e-ac0f-dc29f4a78562 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:33:48.866118 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-abaa88c9-c4cd-405e-ac0f-dc29f4a78562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:48.909194 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce914b69-20ac-4bcb-9391-53b7af436424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:48.910045 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce914b69-20ac-4bcb-9391-53b7af436424 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:48.910468 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce914b69-20ac-4bcb-9391-53b7af436424 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:48.910871 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce914b69-20ac-4bcb-9391-53b7af436424 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:48.912012 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce914b69-20ac-4bcb-9391-53b7af436424 None admin] RBAC: Adding query filter params (role.id=b674fd0a592e4e0cada811bba5ceea4d, user.id=d108dc5b598d48b19e5232719ef33dc1, scope.system=all) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:33:48.912562 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce914b69-20ac-4bcb-9391-53b7af436424 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:48.914167 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce914b69-20ac-4bcb-9391-53b7af436424 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:48.921816 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 142/282] 10.222.0.55 () {56 vars in 1308 bytes} [Wed Aug 24 08:33:48 2022] GET /identity/v3/role_assignments?user.id=d108dc5b598d48b19e5232719ef33dc1&scope.system=all&role.id=b674fd0a592e4e0cada811bba5ceea4d => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:33:48.923040 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce914b69-20ac-4bcb-9391-53b7af436424 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:49.003582 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 141/283] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:33:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 08:33:49.081349 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce914b69-20ac-4bcb-9391-53b7af436424 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:49.115244 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:49.123200 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e294f90-3d29-4339-ad7b-b27d4a373a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:49.124096 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e294f90-3d29-4339-ad7b-b27d4a373a64 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:49.124375 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e294f90-3d29-4339-ad7b-b27d4a373a64 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:49.124676 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e294f90-3d29-4339-ad7b-b27d4a373a64 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:49.137518 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e294f90-3d29-4339-ad7b-b27d4a373a64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:33:49.144234 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-0e294f90-3d29-4339-ad7b-b27d4a373a64 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 08:33:49.144234 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:33:49.144234 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:33:49.144234 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:33:49.144234 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:33:49.144234 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:33:49.144234 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:33:49.144234 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:33:49.144234 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:33:49.144234 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:33:49.144234 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:33:49.144234 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:33:49.144234 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:33:49.144234 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:33:49.144234 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:33:49.144234 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:33:49.145222 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:33:49.145222 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:33:49.145222 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:33:49.145222 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:33:49.145222 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:33:49.145222 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:33:49.145222 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:33:49.145222 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:33:49.145222 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:33:49.145222 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:33:49.145222 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:33:49.145222 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:33:49.145222 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:33:49.145222 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:33:49.145222 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:33:49.145222 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:33:49.146038 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:33:49.146038 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:33:49.146038 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:33:49.146038 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:33:49.146038 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:33:49.146038 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:33:49.146038 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:33:49.146038 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:33:49.146038 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:33:49.146038 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:33:49.146038 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 08:33:49.146038 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:33:49.146642 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 143/284] 10.222.0.55 () {56 vars in 1099 bytes} [Wed Aug 24 08:33:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:33:49.154121 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaa88c9-c4cd-405e-ac0f-dc29f4a78562 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:49.199635 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:49.300056 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaa88c9-c4cd-405e-ac0f-dc29f4a78562 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:49.335803 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f853a0-5286-4ac7-90de-1b55fd7023d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:49.336421 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f853a0-5286-4ac7-90de-1b55fd7023d8 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:49.336745 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f853a0-5286-4ac7-90de-1b55fd7023d8 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:49.337129 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f853a0-5286-4ac7-90de-1b55fd7023d8 None admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:49.337846 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f853a0-5286-4ac7-90de-1b55fd7023d8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:33:49.338647 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f853a0-5286-4ac7-90de-1b55fd7023d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:49.340030 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f853a0-5286-4ac7-90de-1b55fd7023d8 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:49.349253 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 142/285] 10.222.0.55 () {56 vars in 1101 bytes} [Wed Aug 24 08:33:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:49.361701 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f853a0-5286-4ac7-90de-1b55fd7023d8 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:49.362579 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e294f90-3d29-4339-ad7b-b27d4a373a64 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:49.508013 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:49.535251 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f853a0-5286-4ac7-90de-1b55fd7023d8 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:49.557771 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e294f90-3d29-4339-ad7b-b27d4a373a64 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:49.614199 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a9a30d-60f5-42e5-9b3e-dbab531cfd1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:49.614855 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a9a30d-60f5-42e5-9b3e-dbab531cfd1f None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:49.616301 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a9a30d-60f5-42e5-9b3e-dbab531cfd1f None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:49.616608 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a9a30d-60f5-42e5-9b3e-dbab531cfd1f None admin] PATH_INFO: `/v3/users/c63928f678444929bce84f3d0030dc8e` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:49.617956 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a9a30d-60f5-42e5-9b3e-dbab531cfd1f None admin] RBAC: Authorizing `identity:get_user(user_id=c63928f678444929bce84f3d0030dc8e)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:49.619973 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a9a30d-60f5-42e5-9b3e-dbab531cfd1f None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:49.624741 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 144/286] 10.222.0.55 () {56 vars in 1177 bytes} [Wed Aug 24 08:33:49 2022] GET /identity/v3/users/c63928f678444929bce84f3d0030dc8e => generated 297 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:49.628370 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a9a30d-60f5-42e5-9b3e-dbab531cfd1f None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:49.645817 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd313f0c-05cc-4bcc-9fd6-02c9711af49f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:49.646475 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd313f0c-05cc-4bcc-9fd6-02c9711af49f None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:49.646753 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd313f0c-05cc-4bcc-9fd6-02c9711af49f None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:49.647024 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd313f0c-05cc-4bcc-9fd6-02c9711af49f None admin] PATH_INFO: `/v3/roles/member` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:49.669773 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd313f0c-05cc-4bcc-9fd6-02c9711af49f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:33:49.679676 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-dd313f0c-05cc-4bcc-9fd6-02c9711af49f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:33:49.679676 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:33:49.679676 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:33:49.679676 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:33:49.679676 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:33:49.679676 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:33:49.679676 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:33:49.679676 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:33:49.679676 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:33:49.679676 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:33:49.679676 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:33:49.679676 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:33:49.679676 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:33:49.679676 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:33:49.679676 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:33:49.679676 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:33:49.680619 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:33:49.680619 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:33:49.680619 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:33:49.680619 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:33:49.680619 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:33:49.680619 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:33:49.680619 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:33:49.680619 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:33:49.680619 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:33:49.680619 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:33:49.680619 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:33:49.680619 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:33:49.680619 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:33:49.680619 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:33:49.680619 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:33:49.680619 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:33:49.681417 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:33:49.681417 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:33:49.681417 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:33:49.681417 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:33:49.681417 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:33:49.681417 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:33:49.681417 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:33:49.681417 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:33:49.681417 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:33:49.681417 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:33:49.681417 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:33:49.681417 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:33:49.687983 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 143/287] 10.222.0.55 () {56 vars in 1099 bytes} [Wed Aug 24 08:33:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:33:49.689911 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd313f0c-05cc-4bcc-9fd6-02c9711af49f None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:49.755595 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:49.820715 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd313f0c-05cc-4bcc-9fd6-02c9711af49f None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:49.864625 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc648ed-a521-4948-bbd2-9e299cd312af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:49.865671 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc648ed-a521-4948-bbd2-9e299cd312af None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:49.866238 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc648ed-a521-4948-bbd2-9e299cd312af None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:49.866514 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc648ed-a521-4948-bbd2-9e299cd312af None admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:49.867449 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc648ed-a521-4948-bbd2-9e299cd312af None admin] RBAC: Adding query filter params (name=member) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:33:49.867757 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc648ed-a521-4948-bbd2-9e299cd312af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:49.869801 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc648ed-a521-4948-bbd2-9e299cd312af None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:49.875443 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a9a30d-60f5-42e5-9b3e-dbab531cfd1f None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:49.884566 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 144/288] 10.222.0.55 () {56 vars in 1101 bytes} [Wed Aug 24 08:33:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:49.896831 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc648ed-a521-4948-bbd2-9e299cd312af None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:49.924807 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:49.958276 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6041ab92-4a1a-497d-96cf-ea465c9157d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:49.958936 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6041ab92-4a1a-497d-96cf-ea465c9157d8 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:49.959214 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6041ab92-4a1a-497d-96cf-ea465c9157d8 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:49.959484 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6041ab92-4a1a-497d-96cf-ea465c9157d8 None admin] PATH_INFO: `/v3/projects/61247a1049b846b79052f42b0d525259` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:49.962358 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6041ab92-4a1a-497d-96cf-ea465c9157d8 None admin] RBAC: Authorizing `identity:get_project(project_id=61247a1049b846b79052f42b0d525259)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:49.963873 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6041ab92-4a1a-497d-96cf-ea465c9157d8 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:49.970337 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 145/289] 10.222.0.55 () {56 vars in 1186 bytes} [Wed Aug 24 08:33:49 2022] GET /identity/v3/projects/61247a1049b846b79052f42b0d525259 => generated 302 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:49.971487 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6041ab92-4a1a-497d-96cf-ea465c9157d8 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:50.080559 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc648ed-a521-4948-bbd2-9e299cd312af None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:50.146272 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6041ab92-4a1a-497d-96cf-ea465c9157d8 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:50.148009 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77097e5-e600-43d7-8ed4-5e2fd0451928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:50.148880 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77097e5-e600-43d7-8ed4-5e2fd0451928 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:50.149318 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77097e5-e600-43d7-8ed4-5e2fd0451928 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:50.149591 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77097e5-e600-43d7-8ed4-5e2fd0451928 None admin] PATH_INFO: `/v3/users/10ab6b64fbda471a8f7756070346fe3b` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:50.153296 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77097e5-e600-43d7-8ed4-5e2fd0451928 None admin] RBAC: Authorizing `identity:get_user(user_id=10ab6b64fbda471a8f7756070346fe3b)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:50.154781 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77097e5-e600-43d7-8ed4-5e2fd0451928 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:50.156752 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 145/290] 10.222.0.55 () {56 vars in 1177 bytes} [Wed Aug 24 08:33:49 2022] GET /identity/v3/users/10ab6b64fbda471a8f7756070346fe3b => generated 297 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:50.162350 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77097e5-e600-43d7-8ed4-5e2fd0451928 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:50.176718 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:50.208454 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44f572f-220b-4d17-b068-3fb24904ee60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:50.210120 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44f572f-220b-4d17-b068-3fb24904ee60 None admin] REQUEST_METHOD: `PUT` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:50.210633 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44f572f-220b-4d17-b068-3fb24904ee60 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:50.211126 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44f572f-220b-4d17-b068-3fb24904ee60 None admin] PATH_INFO: `/v3/projects/61247a1049b846b79052f42b0d525259/users/c63928f678444929bce84f3d0030dc8e/roles/e308031ba9a84b79bcdf501455ebe0d7` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:50.214046 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44f572f-220b-4d17-b068-3fb24904ee60 None admin] RBAC: Authorizing `identity:create_grant(project_id=61247a1049b846b79052f42b0d525259, user_id=c63928f678444929bce84f3d0030dc8e, role_id=e308031ba9a84b79bcdf501455ebe0d7)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:50.216343 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44f572f-220b-4d17-b068-3fb24904ee60 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:50.299484 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:50.320571 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 146/291] 10.222.0.55 () {58 vars in 1439 bytes} [Wed Aug 24 08:33:49 2022] PUT /identity/v3/projects/61247a1049b846b79052f42b0d525259/users/c63928f678444929bce84f3d0030dc8e/roles/e308031ba9a84b79bcdf501455ebe0d7 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:33:50.323781 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77097e5-e600-43d7-8ed4-5e2fd0451928 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:50.365720 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f18eeca-27bc-46e8-834f-a3eb495f2d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:50.366279 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f18eeca-27bc-46e8-834f-a3eb495f2d5e None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:50.366559 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f18eeca-27bc-46e8-834f-a3eb495f2d5e None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:50.366841 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f18eeca-27bc-46e8-834f-a3eb495f2d5e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:50.448655 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f18eeca-27bc-46e8-834f-a3eb495f2d5e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:50.450214 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f18eeca-27bc-46e8-834f-a3eb495f2d5e None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:50.608319 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:50.663099 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 146/292] 10.222.0.55 () {58 vars in 1282 bytes} [Wed Aug 24 08:33:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 08:33:50.669142 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44f572f-220b-4d17-b068-3fb24904ee60 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:50.793415 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44f572f-220b-4d17-b068-3fb24904ee60 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:50.831920 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a850644f-4e6e-4d3a-9522-c7d0ee95714d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:50.832592 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a850644f-4e6e-4d3a-9522-c7d0ee95714d None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:50.833210 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a850644f-4e6e-4d3a-9522-c7d0ee95714d None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:50.833503 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a850644f-4e6e-4d3a-9522-c7d0ee95714d None admin] PATH_INFO: `/v3/projects/61247a1049b846b79052f42b0d525259` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:50.834740 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a850644f-4e6e-4d3a-9522-c7d0ee95714d None admin] RBAC: Authorizing `identity:get_project(project_id=61247a1049b846b79052f42b0d525259)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:50.836223 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a850644f-4e6e-4d3a-9522-c7d0ee95714d None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:50.838811 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 147/293] 10.222.0.55 () {56 vars in 1186 bytes} [Wed Aug 24 08:33:50 2022] GET /identity/v3/projects/61247a1049b846b79052f42b0d525259 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:50.839730 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906cca02-674a-433d-8ef8-8eacb77bfe63 None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:50.840182 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906cca02-674a-433d-8ef8-8eacb77bfe63 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:50.840784 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906cca02-674a-433d-8ef8-8eacb77bfe63 None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:50.843607 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 147/294] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:33:50 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:33:50.844577 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a850644f-4e6e-4d3a-9522-c7d0ee95714d None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:50.850025 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24734af-255a-4a83-acb1-28c4b391bd87 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:50.850495 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24734af-255a-4a83-acb1-28c4b391bd87 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:50.850989 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24734af-255a-4a83-acb1-28c4b391bd87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:50.886357 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-b24734af-255a-4a83-acb1-28c4b391bd87 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:33:50.887805 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-b24734af-255a-4a83-acb1-28c4b391bd87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:50.948583 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:51.008576 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a850644f-4e6e-4d3a-9522-c7d0ee95714d None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:51.047074 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58078c11-1363-462d-8bfb-547894dc1a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:51.047703 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58078c11-1363-462d-8bfb-547894dc1a7e None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:51.047987 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58078c11-1363-462d-8bfb-547894dc1a7e None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:51.048266 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58078c11-1363-462d-8bfb-547894dc1a7e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:51.049067 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58078c11-1363-462d-8bfb-547894dc1a7e None admin] RBAC: Adding query filter params (role.id=b674fd0a592e4e0cada811bba5ceea4d, scope.project.id=61247a1049b846b79052f42b0d525259, user.id=10ab6b64fbda471a8f7756070346fe3b) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:33:51.049381 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58078c11-1363-462d-8bfb-547894dc1a7e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:51.050860 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58078c11-1363-462d-8bfb-547894dc1a7e None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:51.058221 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 148/295] 10.222.0.55 () {56 vars in 1374 bytes} [Wed Aug 24 08:33:50 2022] GET /identity/v3/role_assignments?user.id=10ab6b64fbda471a8f7756070346fe3b&scope.project.id=61247a1049b846b79052f42b0d525259&role.id=b674fd0a592e4e0cada811bba5ceea4d => generated 608 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:33:51.086565 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 148/296] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:33:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:33:51.266146 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8e25ac-2948-4a1d-80d5-5002e51ade14 None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:51.266362 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8e25ac-2948-4a1d-80d5-5002e51ade14 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:51.266619 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8e25ac-2948-4a1d-80d5-5002e51ade14 None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:51.268471 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 149/297] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:33:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:33:51.274249 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3015edf-3d54-4258-a50f-627a9e2e7fff None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:51.274579 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3015edf-3d54-4258-a50f-627a9e2e7fff None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:51.274803 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3015edf-3d54-4258-a50f-627a9e2e7fff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:51.297535 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-b3015edf-3d54-4258-a50f-627a9e2e7fff None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:33:51.298747 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-b3015edf-3d54-4258-a50f-627a9e2e7fff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:51.323619 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7853dbe-266b-44e0-bde9-73bdbf492130 None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:51.323967 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7853dbe-266b-44e0-bde9-73bdbf492130 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:51.324176 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7853dbe-266b-44e0-bde9-73bdbf492130 None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:51.326870 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 150/298] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:33:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:33:51.331454 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cabe75-0dc7-47aa-8750-85641b1011af None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:51.331743 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cabe75-0dc7-47aa-8750-85641b1011af None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:51.332007 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cabe75-0dc7-47aa-8750-85641b1011af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:51.351886 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-18cabe75-0dc7-47aa-8750-85641b1011af None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:33:51.353162 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-18cabe75-0dc7-47aa-8750-85641b1011af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:51.404102 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 149/299] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:33:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:33:51.436388 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 151/300] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:33:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 08:33:51.597504 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3015edf-3d54-4258-a50f-627a9e2e7fff None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:51.670271 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:51.723153 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cabe75-0dc7-47aa-8750-85641b1011af None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:51.775927 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:51.793462 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3015edf-3d54-4258-a50f-627a9e2e7fff None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:51.840395 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cabe75-0dc7-47aa-8750-85641b1011af None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:51.851089 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1847c454-219a-40f0-a1ad-0a9d7f1a3b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:51.852116 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1847c454-219a-40f0-a1ad-0a9d7f1a3b56 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:51.852663 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1847c454-219a-40f0-a1ad-0a9d7f1a3b56 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:51.853158 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1847c454-219a-40f0-a1ad-0a9d7f1a3b56 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:51.876567 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1847c454-219a-40f0-a1ad-0a9d7f1a3b56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:33:51.880997 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5c5b40-a381-4547-8430-57c0f621bb9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:51.881643 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5c5b40-a381-4547-8430-57c0f621bb9c None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:51.881924 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5c5b40-a381-4547-8430-57c0f621bb9c None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:51.882201 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5c5b40-a381-4547-8430-57c0f621bb9c None admin] PATH_INFO: `/v3/domains/default` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:51.883338 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-1847c454-219a-40f0-a1ad-0a9d7f1a3b56 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 08:33:51.883338 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:33:51.883338 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:33:51.883338 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:33:51.883338 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:33:51.883338 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:33:51.883338 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:33:51.883338 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:33:51.883338 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:33:51.883338 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:33:51.883338 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:33:51.883338 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:33:51.883338 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:33:51.883338 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:33:51.883338 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:33:51.883338 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:33:51.884188 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:33:51.884188 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:33:51.884188 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:33:51.884188 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:33:51.884188 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:33:51.884188 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:33:51.884188 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:33:51.884188 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:33:51.884188 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:33:51.884188 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:33:51.884188 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:33:51.884188 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:33:51.884188 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:33:51.884188 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:33:51.884188 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:33:51.884188 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:33:51.885089 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:33:51.885089 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:33:51.885089 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:33:51.885089 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:33:51.885089 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:33:51.885089 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:33:51.885089 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:33:51.885089 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:33:51.885089 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:33:51.885089 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:33:51.885089 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 08:33:51.885089 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:33:51.885712 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5c5b40-a381-4547-8430-57c0f621bb9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:51.885712 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5c5b40-a381-4547-8430-57c0f621bb9c None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:51.885883 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 150/301] 10.222.0.55 () {56 vars in 1099 bytes} [Wed Aug 24 08:33:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:33:51.886711 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 152/302] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:33:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:51.887312 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1847c454-219a-40f0-a1ad-0a9d7f1a3b56 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:51.891190 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5c5b40-a381-4547-8430-57c0f621bb9c None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:52.015032 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5c5b40-a381-4547-8430-57c0f621bb9c None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:52.029685 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:52.037956 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:52.043168 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1847c454-219a-40f0-a1ad-0a9d7f1a3b56 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:52.051875 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c048d30-be73-4fc8-9410-95682e8d8071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:52.052489 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c048d30-be73-4fc8-9410-95682e8d8071 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:52.053062 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c048d30-be73-4fc8-9410-95682e8d8071 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:52.053062 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c048d30-be73-4fc8-9410-95682e8d8071 None admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:52.053721 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c048d30-be73-4fc8-9410-95682e8d8071 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:33:52.054033 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c048d30-be73-4fc8-9410-95682e8d8071 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:52.055421 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c048d30-be73-4fc8-9410-95682e8d8071 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:52.064093 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 153/303] 10.222.0.55 () {56 vars in 1101 bytes} [Wed Aug 24 08:33:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:33:52.064947 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c048d30-be73-4fc8-9410-95682e8d8071 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:52.081624 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521bfcfa-b067-4f28-8df8-6efdf6714f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:52.082426 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521bfcfa-b067-4f28-8df8-6efdf6714f04 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:52.082714 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521bfcfa-b067-4f28-8df8-6efdf6714f04 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:52.083136 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521bfcfa-b067-4f28-8df8-6efdf6714f04 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:52.096651 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521bfcfa-b067-4f28-8df8-6efdf6714f04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:33:52.102431 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-521bfcfa-b067-4f28-8df8-6efdf6714f04 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:33:52.102431 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:33:52.102431 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:33:52.102431 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:33:52.102431 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:33:52.102431 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:33:52.102431 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:33:52.102431 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:33:52.102431 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:33:52.102431 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:33:52.102431 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:33:52.102431 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:33:52.102431 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:33:52.102431 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:33:52.102431 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:33:52.102431 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:33:52.103374 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:33:52.103374 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:33:52.103374 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:33:52.103374 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:33:52.103374 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:33:52.103374 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:33:52.103374 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:33:52.103374 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:33:52.103374 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:33:52.103374 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:33:52.103374 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:33:52.103374 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:33:52.103374 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:33:52.103374 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:33:52.103374 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:33:52.103374 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:33:52.104754 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:33:52.104754 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:33:52.104754 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:33:52.104754 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:33:52.104754 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:33:52.104754 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:33:52.104754 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:33:52.104754 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:33:52.104754 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:33:52.104754 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:33:52.104754 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:33:52.104754 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:33:52.104754 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 151/304] 10.222.0.55 () {56 vars in 1099 bytes} [Wed Aug 24 08:33:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:33:52.104754 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521bfcfa-b067-4f28-8df8-6efdf6714f04 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:52.224368 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c048d30-be73-4fc8-9410-95682e8d8071 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:52.243304 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521bfcfa-b067-4f28-8df8-6efdf6714f04 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:52.292554 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebe5c93-193c-4b8d-a24e-3710c2c0e1bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:52.293359 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebe5c93-193c-4b8d-a24e-3710c2c0e1bd None admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:52.293853 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebe5c93-193c-4b8d-a24e-3710c2c0e1bd None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:52.293957 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88473f20-1b5b-437c-b520-5bd41bfd3841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:52.294356 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebe5c93-193c-4b8d-a24e-3710c2c0e1bd None admin] PATH_INFO: `/v3/groups` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:52.295052 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88473f20-1b5b-437c-b520-5bd41bfd3841 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:52.295547 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88473f20-1b5b-437c-b520-5bd41bfd3841 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:52.295645 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebe5c93-193c-4b8d-a24e-3710c2c0e1bd None admin] RBAC: Authorizing `identity:create_group()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:52.296038 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88473f20-1b5b-437c-b520-5bd41bfd3841 None admin] PATH_INFO: `/v3/users/73ee7493ec2f4d019cd87308b7d19514` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:52.298480 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebe5c93-193c-4b8d-a24e-3710c2c0e1bd None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:52.317197 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88473f20-1b5b-437c-b520-5bd41bfd3841 None admin] RBAC: Authorizing `identity:get_user(user_id=73ee7493ec2f4d019cd87308b7d19514)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:52.319037 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88473f20-1b5b-437c-b520-5bd41bfd3841 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:52.321400 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 152/305] 10.222.0.55 () {56 vars in 1177 bytes} [Wed Aug 24 08:33:52 2022] GET /identity/v3/users/73ee7493ec2f4d019cd87308b7d19514 => generated 293 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:52.324774 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88473f20-1b5b-437c-b520-5bd41bfd3841 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:52.359616 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:52.392978 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 154/306] 10.222.0.55 () {60 vars in 1134 bytes} [Wed Aug 24 08:33:52 2022] POST /identity/v3/groups => generated 229 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:33:52.395035 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebe5c93-193c-4b8d-a24e-3710c2c0e1bd None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:52.452743 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:52.482489 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88473f20-1b5b-437c-b520-5bd41bfd3841 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:52.525766 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e813845-d807-4c44-af96-442c1e60830d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:52.526779 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e813845-d807-4c44-af96-442c1e60830d None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:52.527231 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e813845-d807-4c44-af96-442c1e60830d None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:52.527695 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e813845-d807-4c44-af96-442c1e60830d None admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:52.528577 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e813845-d807-4c44-af96-442c1e60830d None admin] RBAC: Adding query filter params (name=member) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:33:52.529104 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e813845-d807-4c44-af96-442c1e60830d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:52.530737 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e813845-d807-4c44-af96-442c1e60830d None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:52.531046 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebe5c93-193c-4b8d-a24e-3710c2c0e1bd None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:52.546627 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 153/307] 10.222.0.55 () {56 vars in 1101 bytes} [Wed Aug 24 08:33:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:52.547920 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e813845-d807-4c44-af96-442c1e60830d None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:52.575339 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad270f2-f3f7-40b8-a7c2-4b2a47a5ad9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:52.575942 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad270f2-f3f7-40b8-a7c2-4b2a47a5ad9a None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:52.576315 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad270f2-f3f7-40b8-a7c2-4b2a47a5ad9a None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:52.576871 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad270f2-f3f7-40b8-a7c2-4b2a47a5ad9a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:52.578242 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad270f2-f3f7-40b8-a7c2-4b2a47a5ad9a None admin] RBAC: Adding query filter params (role.id=e308031ba9a84b79bcdf501455ebe0d7, user.id=73ee7493ec2f4d019cd87308b7d19514, scope.system=all) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:33:52.578783 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad270f2-f3f7-40b8-a7c2-4b2a47a5ad9a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:52.581502 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad270f2-f3f7-40b8-a7c2-4b2a47a5ad9a None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:52.591537 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 155/308] 10.222.0.55 () {56 vars in 1308 bytes} [Wed Aug 24 08:33:52 2022] GET /identity/v3/role_assignments?user.id=73ee7493ec2f4d019cd87308b7d19514&scope.system=all&role.id=e308031ba9a84b79bcdf501455ebe0d7 => generated 226 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:52.686643 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:52.727550 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e813845-d807-4c44-af96-442c1e60830d None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:52.787170 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896e063a-0789-49a6-b7c9-b21e73df6d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:52.788081 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896e063a-0789-49a6-b7c9-b21e73df6d7d None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:52.788502 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896e063a-0789-49a6-b7c9-b21e73df6d7d None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:52.788927 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896e063a-0789-49a6-b7c9-b21e73df6d7d None admin] PATH_INFO: `/v3/users/d108dc5b598d48b19e5232719ef33dc1` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:52.790283 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896e063a-0789-49a6-b7c9-b21e73df6d7d None admin] RBAC: Authorizing `identity:get_user(user_id=d108dc5b598d48b19e5232719ef33dc1)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:52.793343 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896e063a-0789-49a6-b7c9-b21e73df6d7d None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:52.796705 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 154/309] 10.222.0.55 () {56 vars in 1177 bytes} [Wed Aug 24 08:33:52 2022] GET /identity/v3/users/d108dc5b598d48b19e5232719ef33dc1 => generated 293 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:52.802413 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad270f2-f3f7-40b8-a7c2-4b2a47a5ad9a None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:52.915964 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:52.919959 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad270f2-f3f7-40b8-a7c2-4b2a47a5ad9a None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:52.958752 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be1d207-2ef7-440d-b51f-aafcb36f79db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:52.959328 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be1d207-2ef7-440d-b51f-aafcb36f79db None admin] REQUEST_METHOD: `PUT` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:52.959606 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be1d207-2ef7-440d-b51f-aafcb36f79db None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:52.959879 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be1d207-2ef7-440d-b51f-aafcb36f79db None admin] PATH_INFO: `/v3/system/users/d108dc5b598d48b19e5232719ef33dc1/roles/b674fd0a592e4e0cada811bba5ceea4d` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:52.961406 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be1d207-2ef7-440d-b51f-aafcb36f79db None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=d108dc5b598d48b19e5232719ef33dc1, role_id=b674fd0a592e4e0cada811bba5ceea4d)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:52.962793 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be1d207-2ef7-440d-b51f-aafcb36f79db None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:52.966128 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ed61d0-37e8-489c-af35-978b739e53d1 None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:52.966412 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ed61d0-37e8-489c-af35-978b739e53d1 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:52.966695 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ed61d0-37e8-489c-af35-978b739e53d1 None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:52.968968 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 155/310] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:33:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:33:52.974327 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44bcee8-4ae5-446a-9eb7-f2213758da3c None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:52.974612 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44bcee8-4ae5-446a-9eb7-f2213758da3c None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:52.974933 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44bcee8-4ae5-446a-9eb7-f2213758da3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:52.994471 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-e44bcee8-4ae5-446a-9eb7-f2213758da3c None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:33:52.995741 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-e44bcee8-4ae5-446a-9eb7-f2213758da3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:53.068125 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 156/311] 10.222.0.55 () {58 vars in 1334 bytes} [Wed Aug 24 08:33:52 2022] PUT /identity/v3/system/users/d108dc5b598d48b19e5232719ef33dc1/roles/b674fd0a592e4e0cada811bba5ceea4d => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:33:53.091013 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 156/312] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:33:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 08:33:53.586639 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be1d207-2ef7-440d-b51f-aafcb36f79db None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:53.717381 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be1d207-2ef7-440d-b51f-aafcb36f79db None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:53.756277 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f33340-6a84-4a43-a3c9-eb58e4e7de63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:53.757259 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f33340-6a84-4a43-a3c9-eb58e4e7de63 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:53.757715 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f33340-6a84-4a43-a3c9-eb58e4e7de63 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:53.758161 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f33340-6a84-4a43-a3c9-eb58e4e7de63 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:53.769889 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f33340-6a84-4a43-a3c9-eb58e4e7de63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:33:53.775558 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-78f33340-6a84-4a43-a3c9-eb58e4e7de63 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 08:33:53.775558 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:33:53.775558 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:33:53.775558 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:33:53.775558 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:33:53.775558 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:33:53.775558 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:33:53.775558 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:33:53.775558 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:33:53.775558 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:33:53.775558 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:33:53.775558 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:33:53.775558 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:33:53.775558 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:33:53.775558 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:33:53.775558 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:33:53.776528 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:33:53.776528 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:33:53.776528 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:33:53.776528 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:33:53.776528 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:33:53.776528 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:33:53.776528 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:33:53.776528 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:33:53.776528 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:33:53.776528 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:33:53.776528 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:33:53.776528 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:33:53.776528 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:33:53.776528 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:33:53.776528 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:33:53.776528 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:33:53.777337 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:33:53.777337 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:33:53.777337 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:33:53.777337 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:33:53.777337 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:33:53.777337 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:33:53.777337 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:33:53.777337 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:33:53.777337 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:33:53.777337 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:33:53.777337 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 08:33:53.777337 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:33:53.779713 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 157/313] 10.222.0.55 () {56 vars in 1099 bytes} [Wed Aug 24 08:33:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:33:53.789533 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44bcee8-4ae5-446a-9eb7-f2213758da3c None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:53.893209 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:53.915625 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44bcee8-4ae5-446a-9eb7-f2213758da3c None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:53.960948 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a3ef9f-dfd5-4dcc-95ad-d9f519b26e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:53.961891 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a3ef9f-dfd5-4dcc-95ad-d9f519b26e7c None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:53.962311 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a3ef9f-dfd5-4dcc-95ad-d9f519b26e7c None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:53.962746 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a3ef9f-dfd5-4dcc-95ad-d9f519b26e7c None admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:53.964146 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a3ef9f-dfd5-4dcc-95ad-d9f519b26e7c None admin] RBAC: Adding query filter params (name=reader) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:33:53.964609 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a3ef9f-dfd5-4dcc-95ad-d9f519b26e7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:53.966173 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a3ef9f-dfd5-4dcc-95ad-d9f519b26e7c None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:53.981036 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 157/314] 10.222.0.55 () {56 vars in 1101 bytes} [Wed Aug 24 08:33:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:53.983629 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f33340-6a84-4a43-a3c9-eb58e4e7de63 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:54.013390 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:54.107209 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f33340-6a84-4a43-a3c9-eb58e4e7de63 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:54.148479 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9a982f-608e-4043-9d75-8fb3c212fc19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:54.149072 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9a982f-608e-4043-9d75-8fb3c212fc19 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:54.149355 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9a982f-608e-4043-9d75-8fb3c212fc19 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:54.149632 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9a982f-608e-4043-9d75-8fb3c212fc19 None admin] PATH_INFO: `/v3/users/c63928f678444929bce84f3d0030dc8e` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:54.151327 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9a982f-608e-4043-9d75-8fb3c212fc19 None admin] RBAC: Authorizing `identity:get_user(user_id=c63928f678444929bce84f3d0030dc8e)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:54.152856 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9a982f-608e-4043-9d75-8fb3c212fc19 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:54.154959 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 158/315] 10.222.0.55 () {56 vars in 1177 bytes} [Wed Aug 24 08:33:53 2022] GET /identity/v3/users/c63928f678444929bce84f3d0030dc8e => generated 297 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:54.164289 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a3ef9f-dfd5-4dcc-95ad-d9f519b26e7c None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:54.352034 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a3ef9f-dfd5-4dcc-95ad-d9f519b26e7c None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:54.410519 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1f3b4b-2a9a-4a1c-9899-6ac6fe30b600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:54.411063 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1f3b4b-2a9a-4a1c-9899-6ac6fe30b600 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:54.411396 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1f3b4b-2a9a-4a1c-9899-6ac6fe30b600 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:54.411669 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1f3b4b-2a9a-4a1c-9899-6ac6fe30b600 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:54.421372 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:54.489037 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1f3b4b-2a9a-4a1c-9899-6ac6fe30b600 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:54.490491 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1f3b4b-2a9a-4a1c-9899-6ac6fe30b600 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:54.600847 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 158/316] 10.222.0.55 () {58 vars in 1282 bytes} [Wed Aug 24 08:33:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:33:54.606745 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9a982f-608e-4043-9d75-8fb3c212fc19 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:54.720111 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d744d70-c074-47e9-b654-9f8f5ab27ce5 None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:54.720330 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:54.720444 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d744d70-c074-47e9-b654-9f8f5ab27ce5 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:54.720624 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d744d70-c074-47e9-b654-9f8f5ab27ce5 None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:54.722618 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 159/317] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:33:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:33:54.727905 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bbc5f0-eb2b-4db3-8d60-d693c005be08 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:54.728184 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bbc5f0-eb2b-4db3-8d60-d693c005be08 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:54.728671 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bbc5f0-eb2b-4db3-8d60-d693c005be08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:54.732605 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9a982f-608e-4043-9d75-8fb3c212fc19 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:54.747622 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-96bbc5f0-eb2b-4db3-8d60-d693c005be08 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:33:54.748932 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-96bbc5f0-eb2b-4db3-8d60-d693c005be08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:54.773034 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2c6d08-144d-41f8-8b53-0b2a2d64e36d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:54.773681 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2c6d08-144d-41f8-8b53-0b2a2d64e36d None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:54.773963 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2c6d08-144d-41f8-8b53-0b2a2d64e36d None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:54.774239 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2c6d08-144d-41f8-8b53-0b2a2d64e36d None admin] PATH_INFO: `/v3/projects/61247a1049b846b79052f42b0d525259` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:54.775503 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2c6d08-144d-41f8-8b53-0b2a2d64e36d None admin] RBAC: Authorizing `identity:get_project(project_id=61247a1049b846b79052f42b0d525259)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:54.777612 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2c6d08-144d-41f8-8b53-0b2a2d64e36d None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:54.779793 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 159/318] 10.222.0.55 () {56 vars in 1186 bytes} [Wed Aug 24 08:33:54 2022] GET /identity/v3/projects/61247a1049b846b79052f42b0d525259 => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:54.784745 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:54.786631 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2c6d08-144d-41f8-8b53-0b2a2d64e36d None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:54.832852 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 160/319] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:33:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 08:33:54.903237 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2c6d08-144d-41f8-8b53-0b2a2d64e36d None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:54.941177 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b56a9b-928c-4a3c-b30d-8fa6d149b586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:54.941792 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b56a9b-928c-4a3c-b30d-8fa6d149b586 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:54.942230 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b56a9b-928c-4a3c-b30d-8fa6d149b586 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:54.942507 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b56a9b-928c-4a3c-b30d-8fa6d149b586 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:54.943308 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b56a9b-928c-4a3c-b30d-8fa6d149b586 None admin] RBAC: Adding query filter params (role.id=e308031ba9a84b79bcdf501455ebe0d7, scope.project.id=61247a1049b846b79052f42b0d525259, user.id=c63928f678444929bce84f3d0030dc8e) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:33:54.943661 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b56a9b-928c-4a3c-b30d-8fa6d149b586 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:54.945128 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b56a9b-928c-4a3c-b30d-8fa6d149b586 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:54.952008 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 160/320] 10.222.0.55 () {56 vars in 1374 bytes} [Wed Aug 24 08:33:54 2022] GET /identity/v3/role_assignments?user.id=c63928f678444929bce84f3d0030dc8e&scope.project.id=61247a1049b846b79052f42b0d525259&role.id=e308031ba9a84b79bcdf501455ebe0d7 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:33:55.065400 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d961b48-e622-4ac9-adef-ce7cb79aed3d None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:55.065774 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d961b48-e622-4ac9-adef-ce7cb79aed3d None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:55.065949 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d961b48-e622-4ac9-adef-ce7cb79aed3d None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:55.067711 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 161/321] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:33:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:33:55.075122 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1922d19b-2302-4041-ad03-c29ad361a779 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:55.075686 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1922d19b-2302-4041-ad03-c29ad361a779 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:55.075686 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1922d19b-2302-4041-ad03-c29ad361a779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:55.095786 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-1922d19b-2302-4041-ad03-c29ad361a779 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:33:55.097056 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-1922d19b-2302-4041-ad03-c29ad361a779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:55.099516 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d961b48-e622-4ac9-adef-ce7cb79aed3d None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:55.117665 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:55.184582 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 161/322] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:33:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:33:55.185465 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe3cd24-94b0-4063-9ab2-2f33a6686137 None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:55.185749 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe3cd24-94b0-4063-9ab2-2f33a6686137 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:55.186017 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe3cd24-94b0-4063-9ab2-2f33a6686137 None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:55.187250 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 162/323] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:33:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:33:55.194655 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b04acc-fd86-4db2-ba8f-faf69529ab30 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:55.194940 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b04acc-fd86-4db2-ba8f-faf69529ab30 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:55.195214 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b04acc-fd86-4db2-ba8f-faf69529ab30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:55.217649 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-b8b04acc-fd86-4db2-ba8f-faf69529ab30 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:33:55.218355 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-b8b04acc-fd86-4db2-ba8f-faf69529ab30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:55.244023 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d961b48-e622-4ac9-adef-ce7cb79aed3d None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:55.285096 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25737544-8488-406a-a2d5-eb9a377de9ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:55.285825 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25737544-8488-406a-a2d5-eb9a377de9ef None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:55.285998 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25737544-8488-406a-a2d5-eb9a377de9ef None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:55.286279 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25737544-8488-406a-a2d5-eb9a377de9ef None admin] PATH_INFO: `/v3/domains/default` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:55.287418 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25737544-8488-406a-a2d5-eb9a377de9ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:55.288875 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25737544-8488-406a-a2d5-eb9a377de9ef None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:55.290582 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 162/324] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:33:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:55.295869 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25737544-8488-406a-a2d5-eb9a377de9ef None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:55.301208 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 163/325] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:33:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 08:33:55.335712 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:55.405278 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25737544-8488-406a-a2d5-eb9a377de9ef None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:55.445782 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbb7345-a203-4865-a918-3bec17887f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:55.446452 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbb7345-a203-4865-a918-3bec17887f75 None admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:55.448175 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbb7345-a203-4865-a918-3bec17887f75 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:55.448175 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbb7345-a203-4865-a918-3bec17887f75 None admin] PATH_INFO: `/v3/groups` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:55.448819 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbb7345-a203-4865-a918-3bec17887f75 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:55.450430 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbb7345-a203-4865-a918-3bec17887f75 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:55.451818 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b04acc-fd86-4db2-ba8f-faf69529ab30 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:55.538243 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:55.545024 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 163/326] 10.222.0.55 () {60 vars in 1134 bytes} [Wed Aug 24 08:33:55 2022] POST /identity/v3/groups => generated 226 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:33:55.565979 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b04acc-fd86-4db2-ba8f-faf69529ab30 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:55.578822 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbb7345-a203-4865-a918-3bec17887f75 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:55.606490 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0917b8aa-e692-4011-9fe1-cbf6c5fd85bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:55.607244 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0917b8aa-e692-4011-9fe1-cbf6c5fd85bb None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:55.607502 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0917b8aa-e692-4011-9fe1-cbf6c5fd85bb None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:55.607760 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0917b8aa-e692-4011-9fe1-cbf6c5fd85bb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:55.622319 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0917b8aa-e692-4011-9fe1-cbf6c5fd85bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:33:55.628837 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-0917b8aa-e692-4011-9fe1-cbf6c5fd85bb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 08:33:55.628837 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:33:55.628837 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:33:55.628837 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:33:55.628837 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:33:55.628837 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:33:55.628837 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:33:55.628837 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:33:55.628837 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:33:55.628837 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:33:55.628837 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:33:55.628837 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:33:55.628837 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:33:55.628837 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:33:55.628837 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:33:55.628837 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:33:55.629831 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:33:55.629831 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:33:55.629831 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:33:55.629831 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:33:55.629831 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:33:55.629831 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:33:55.629831 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:33:55.629831 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:33:55.629831 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:33:55.629831 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:33:55.629831 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:33:55.629831 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:33:55.629831 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:33:55.629831 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:33:55.629831 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:33:55.629831 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:33:55.631219 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:33:55.631219 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:33:55.631219 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:33:55.631219 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:33:55.631219 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:33:55.631219 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:33:55.631219 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:33:55.631219 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:33:55.631219 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:33:55.631219 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:33:55.631219 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 08:33:55.631219 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:33:55.631219 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 164/327] 10.222.0.55 () {56 vars in 1099 bytes} [Wed Aug 24 08:33:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:33:55.636508 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0917b8aa-e692-4011-9fe1-cbf6c5fd85bb None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:55.697472 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:55.703396 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbb7345-a203-4865-a918-3bec17887f75 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:55.753184 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3797660f-ff46-43de-8275-1ba681225bce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:55.753832 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3797660f-ff46-43de-8275-1ba681225bce None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:55.754108 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3797660f-ff46-43de-8275-1ba681225bce None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:55.754489 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3797660f-ff46-43de-8275-1ba681225bce None admin] PATH_INFO: `/v3/roles/member` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:55.771445 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3797660f-ff46-43de-8275-1ba681225bce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:33:55.777031 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0917b8aa-e692-4011-9fe1-cbf6c5fd85bb None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:55.780830 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-3797660f-ff46-43de-8275-1ba681225bce None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:33:55.780830 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:33:55.780830 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:33:55.780830 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:33:55.780830 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:33:55.780830 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:33:55.780830 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:33:55.780830 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:33:55.780830 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:33:55.780830 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:33:55.780830 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:33:55.780830 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:33:55.780830 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:33:55.780830 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:33:55.780830 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:33:55.780830 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:33:55.784372 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:33:55.784372 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:33:55.784372 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:33:55.784372 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:33:55.784372 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:33:55.784372 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:33:55.784372 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:33:55.784372 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:33:55.784372 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:33:55.784372 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:33:55.784372 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:33:55.784372 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:33:55.784372 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:33:55.784372 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:33:55.784372 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:33:55.784372 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:33:55.785265 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:33:55.785265 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:33:55.785265 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:33:55.785265 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:33:55.785265 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:33:55.785265 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:33:55.785265 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:33:55.785265 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:33:55.785265 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:33:55.785265 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:33:55.785265 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:33:55.785265 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:33:55.785265 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 164/328] 10.222.0.55 () {56 vars in 1099 bytes} [Wed Aug 24 08:33:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:33:55.791113 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3797660f-ff46-43de-8275-1ba681225bce None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:55.824806 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9501b8-e767-413b-9ba9-ac2b2b2438f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:55.826045 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9501b8-e767-413b-9ba9-ac2b2b2438f7 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:55.826478 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9501b8-e767-413b-9ba9-ac2b2b2438f7 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:55.826911 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9501b8-e767-413b-9ba9-ac2b2b2438f7 None admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:55.827761 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9501b8-e767-413b-9ba9-ac2b2b2438f7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:33:55.828066 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9501b8-e767-413b-9ba9-ac2b2b2438f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:55.830178 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9501b8-e767-413b-9ba9-ac2b2b2438f7 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:55.843000 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 165/329] 10.222.0.55 () {56 vars in 1101 bytes} [Wed Aug 24 08:33:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:33:55.849155 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9501b8-e767-413b-9ba9-ac2b2b2438f7 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:55.855094 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:55.966497 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3797660f-ff46-43de-8275-1ba681225bce None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:56.011246 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9501b8-e767-413b-9ba9-ac2b2b2438f7 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:56.017594 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9719553a-2668-4d0b-8ab6-e16751f98c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:56.018220 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9719553a-2668-4d0b-8ab6-e16751f98c48 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:56.018503 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9719553a-2668-4d0b-8ab6-e16751f98c48 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:56.018778 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9719553a-2668-4d0b-8ab6-e16751f98c48 None admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:56.019475 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9719553a-2668-4d0b-8ab6-e16751f98c48 None admin] RBAC: Adding query filter params (name=member) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:33:56.019904 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9719553a-2668-4d0b-8ab6-e16751f98c48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:56.021420 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9719553a-2668-4d0b-8ab6-e16751f98c48 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:56.033936 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 165/330] 10.222.0.55 () {56 vars in 1101 bytes} [Wed Aug 24 08:33:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:56.039329 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9719553a-2668-4d0b-8ab6-e16751f98c48 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:56.060625 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b925f2-1565-4a77-9e6d-36498183f08d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:56.061283 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b925f2-1565-4a77-9e6d-36498183f08d None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:56.061565 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b925f2-1565-4a77-9e6d-36498183f08d None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:56.061841 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b925f2-1565-4a77-9e6d-36498183f08d None admin] PATH_INFO: `/v3/users/73ee7493ec2f4d019cd87308b7d19514` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:56.063178 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b925f2-1565-4a77-9e6d-36498183f08d None admin] RBAC: Authorizing `identity:get_user(user_id=73ee7493ec2f4d019cd87308b7d19514)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:56.064722 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b925f2-1565-4a77-9e6d-36498183f08d None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:56.066756 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 166/331] 10.222.0.55 () {56 vars in 1177 bytes} [Wed Aug 24 08:33:55 2022] GET /identity/v3/users/73ee7493ec2f4d019cd87308b7d19514 => generated 293 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:56.072975 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:56.074855 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b925f2-1565-4a77-9e6d-36498183f08d None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:56.188916 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:56.192037 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9719553a-2668-4d0b-8ab6-e16751f98c48 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:56.227394 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b925f2-1565-4a77-9e6d-36498183f08d None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:56.261491 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f9db2f-dfc6-4459-a167-6075b9dbff11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:56.262353 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f9db2f-dfc6-4459-a167-6075b9dbff11 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:56.262705 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f9db2f-dfc6-4459-a167-6075b9dbff11 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:56.263051 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f9db2f-dfc6-4459-a167-6075b9dbff11 None admin] PATH_INFO: `/v3/users/d108dc5b598d48b19e5232719ef33dc1` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:56.264527 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f9db2f-dfc6-4459-a167-6075b9dbff11 None admin] RBAC: Authorizing `identity:get_user(user_id=d108dc5b598d48b19e5232719ef33dc1)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:56.266346 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f9db2f-dfc6-4459-a167-6075b9dbff11 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:56.268826 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 166/332] 10.222.0.55 () {56 vars in 1177 bytes} [Wed Aug 24 08:33:56 2022] GET /identity/v3/users/d108dc5b598d48b19e5232719ef33dc1 => generated 293 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:33:56.276776 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f9db2f-dfc6-4459-a167-6075b9dbff11 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:56.277609 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3cf70b-1b2b-4340-aab3-d45ac5ff644c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:56.278510 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3cf70b-1b2b-4340-aab3-d45ac5ff644c None admin] REQUEST_METHOD: `PUT` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:56.278929 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3cf70b-1b2b-4340-aab3-d45ac5ff644c None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:56.279342 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3cf70b-1b2b-4340-aab3-d45ac5ff644c None admin] PATH_INFO: `/v3/system/users/73ee7493ec2f4d019cd87308b7d19514/roles/e308031ba9a84b79bcdf501455ebe0d7` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:56.281693 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3cf70b-1b2b-4340-aab3-d45ac5ff644c None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=73ee7493ec2f4d019cd87308b7d19514, role_id=e308031ba9a84b79bcdf501455ebe0d7)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:56.283311 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3cf70b-1b2b-4340-aab3-d45ac5ff644c None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:56.358486 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 167/333] 10.222.0.55 () {58 vars in 1334 bytes} [Wed Aug 24 08:33:56 2022] PUT /identity/v3/system/users/73ee7493ec2f4d019cd87308b7d19514/roles/e308031ba9a84b79bcdf501455ebe0d7 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:33:56.391878 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:56.409440 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f9db2f-dfc6-4459-a167-6075b9dbff11 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:56.450928 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1c406a-6822-4bc5-9cb7-30c38f829e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:56.451567 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1c406a-6822-4bc5-9cb7-30c38f829e7f None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:56.451845 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1c406a-6822-4bc5-9cb7-30c38f829e7f None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:56.452119 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1c406a-6822-4bc5-9cb7-30c38f829e7f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:56.452909 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1c406a-6822-4bc5-9cb7-30c38f829e7f None admin] RBAC: Adding query filter params (role.id=b674fd0a592e4e0cada811bba5ceea4d, user.id=d108dc5b598d48b19e5232719ef33dc1, scope.system=all) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:33:56.453216 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1c406a-6822-4bc5-9cb7-30c38f829e7f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:56.454689 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1c406a-6822-4bc5-9cb7-30c38f829e7f None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:56.469824 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 167/334] 10.222.0.55 () {56 vars in 1308 bytes} [Wed Aug 24 08:33:56 2022] GET /identity/v3/role_assignments?user.id=d108dc5b598d48b19e5232719ef33dc1&scope.system=all&role.id=b674fd0a592e4e0cada811bba5ceea4d => generated 510 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:33:58.093719 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccc3389-57f3-4b53-a06c-6db65f35019f None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:58.094678 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccc3389-57f3-4b53-a06c-6db65f35019f None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:58.095164 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccc3389-57f3-4b53-a06c-6db65f35019f None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:58.096999 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 168/335] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:33:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:33:58.110236 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc192b85-62e1-455d-8168-17cf0c6ff2ea None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:58.111137 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc192b85-62e1-455d-8168-17cf0c6ff2ea None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:58.111625 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc192b85-62e1-455d-8168-17cf0c6ff2ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:58.133605 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-fc192b85-62e1-455d-8168-17cf0c6ff2ea None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:33:58.135664 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-fc192b85-62e1-455d-8168-17cf0c6ff2ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:58.227129 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 168/336] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:33:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:33:58.388351 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3a8ba5-f134-46aa-a774-0619f177907a None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:58.388979 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3a8ba5-f134-46aa-a774-0619f177907a None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:58.389458 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3a8ba5-f134-46aa-a774-0619f177907a None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:58.391242 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 169/337] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:33:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:33:58.400716 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc5cc71-7679-43c5-89ba-39fe2b2e88ae None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:58.401566 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc5cc71-7679-43c5-89ba-39fe2b2e88ae None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:58.401816 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc5cc71-7679-43c5-89ba-39fe2b2e88ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:58.424926 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-dbc5cc71-7679-43c5-89ba-39fe2b2e88ae None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:33:58.426103 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-dbc5cc71-7679-43c5-89ba-39fe2b2e88ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:58.523033 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 169/338] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:33:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 08:33:58.543342 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3a8ba5-f134-46aa-a774-0619f177907a None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:58.694082 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3a8ba5-f134-46aa-a774-0619f177907a None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:58.739677 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b07031-97ff-480a-b38c-100ae4855843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:58.740727 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b07031-97ff-480a-b38c-100ae4855843 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:58.741226 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b07031-97ff-480a-b38c-100ae4855843 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:58.741734 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b07031-97ff-480a-b38c-100ae4855843 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:58.749800 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:58.759781 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b07031-97ff-480a-b38c-100ae4855843 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:33:58.765467 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-90b07031-97ff-480a-b38c-100ae4855843 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:33:58.765467 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:33:58.765467 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:33:58.765467 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:33:58.765467 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:33:58.765467 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:33:58.765467 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:33:58.765467 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:33:58.765467 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:33:58.765467 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:33:58.765467 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:33:58.765467 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:33:58.765467 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:33:58.765467 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:33:58.765467 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:33:58.765467 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:33:58.768269 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:33:58.768269 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:33:58.768269 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:33:58.768269 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:33:58.768269 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:33:58.768269 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:33:58.768269 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:33:58.768269 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:33:58.768269 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:33:58.768269 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:33:58.768269 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:33:58.768269 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:33:58.768269 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:33:58.768269 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:33:58.768269 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:33:58.768269 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:33:58.769122 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:33:58.769122 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:33:58.769122 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:33:58.769122 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:33:58.769122 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:33:58.769122 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:33:58.769122 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:33:58.769122 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:33:58.769122 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:33:58.769122 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:33:58.769122 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:33:58.769122 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:33:58.769122 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 170/339] 10.222.0.55 () {56 vars in 1099 bytes} [Wed Aug 24 08:33:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:33:58.773620 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc5cc71-7679-43c5-89ba-39fe2b2e88ae None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:58.781220 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:58.930717 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc5cc71-7679-43c5-89ba-39fe2b2e88ae None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:58.971454 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237c4de5-1151-4e69-9381-f1e361c274b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:58.971454 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237c4de5-1151-4e69-9381-f1e361c274b6 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:58.971454 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237c4de5-1151-4e69-9381-f1e361c274b6 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:58.971454 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237c4de5-1151-4e69-9381-f1e361c274b6 None admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:58.971839 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237c4de5-1151-4e69-9381-f1e361c274b6 None admin] RBAC: Adding query filter params (name=member) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:33:58.971839 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237c4de5-1151-4e69-9381-f1e361c274b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:58.971839 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237c4de5-1151-4e69-9381-f1e361c274b6 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:58.985127 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 170/340] 10.222.0.55 () {56 vars in 1101 bytes} [Wed Aug 24 08:33:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:58.990494 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b07031-97ff-480a-b38c-100ae4855843 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:59.131536 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b07031-97ff-480a-b38c-100ae4855843 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:59.173624 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df420b53-d75a-4990-8896-7661d7ebae83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:59.174284 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df420b53-d75a-4990-8896-7661d7ebae83 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:59.174567 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df420b53-d75a-4990-8896-7661d7ebae83 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:59.174840 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df420b53-d75a-4990-8896-7661d7ebae83 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:59.201242 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e9f4b9-9a31-4ee6-b5b3-95501f8ced65 None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:59.202148 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e9f4b9-9a31-4ee6-b5b3-95501f8ced65 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:59.202649 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e9f4b9-9a31-4ee6-b5b3-95501f8ced65 None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:59.204460 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 171/341] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:33:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:33:59.210283 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49087c0-c360-4f7f-959c-62868254a691 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:59.210856 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49087c0-c360-4f7f-959c-62868254a691 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:59.211334 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49087c0-c360-4f7f-959c-62868254a691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:59.251104 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-e49087c0-c360-4f7f-959c-62868254a691 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:33:59.252669 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-e49087c0-c360-4f7f-959c-62868254a691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:59.263915 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:59.271140 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df420b53-d75a-4990-8896-7661d7ebae83 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:59.272836 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df420b53-d75a-4990-8896-7661d7ebae83 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:59.377717 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:59.390754 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 172/342] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:33:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 08:33:59.391724 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49087c0-c360-4f7f-959c-62868254a691 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:59.441554 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 171/343] 10.222.0.55 () {58 vars in 1282 bytes} [Wed Aug 24 08:33:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:33:59.448438 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df420b53-d75a-4990-8896-7661d7ebae83 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:59.537351 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49087c0-c360-4f7f-959c-62868254a691 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:59.584682 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:59.586555 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c330f05-68c3-4b4c-ad11-1779af475f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:59.587439 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c330f05-68c3-4b4c-ad11-1779af475f15 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:59.587881 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c330f05-68c3-4b4c-ad11-1779af475f15 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:59.588302 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c330f05-68c3-4b4c-ad11-1779af475f15 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:59.592473 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df420b53-d75a-4990-8896-7661d7ebae83 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:59.608811 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c330f05-68c3-4b4c-ad11-1779af475f15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:33:59.613915 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-0c330f05-68c3-4b4c-ad11-1779af475f15 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 08:33:59.613915 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:33:59.613915 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:33:59.613915 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:33:59.613915 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:33:59.613915 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:33:59.613915 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:33:59.613915 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:33:59.613915 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:33:59.613915 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:33:59.613915 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:33:59.613915 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:33:59.613915 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:33:59.613915 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:33:59.613915 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:33:59.613915 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:33:59.615026 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:33:59.615026 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:33:59.615026 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:33:59.615026 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:33:59.615026 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:33:59.615026 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:33:59.615026 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:33:59.615026 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:33:59.615026 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:33:59.615026 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:33:59.615026 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:33:59.615026 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:33:59.615026 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:33:59.615026 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:33:59.615026 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:33:59.615026 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:33:59.615871 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:33:59.615871 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:33:59.615871 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:33:59.615871 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:33:59.615871 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:33:59.615871 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:33:59.615871 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:33:59.615871 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:33:59.615871 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:33:59.615871 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:33:59.615871 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 08:33:59.615871 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:33:59.617820 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 173/344] 10.222.0.55 () {56 vars in 1114 bytes} [Wed Aug 24 08:33:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:33:59.619260 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65417d11-541b-45e3-954f-1a2301178bd9 None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:59.619712 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65417d11-541b-45e3-954f-1a2301178bd9 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:59.620105 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65417d11-541b-45e3-954f-1a2301178bd9 None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:59.621845 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 174/345] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:33:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:33:59.627405 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e938d4b-8a9b-4a2f-ae60-a10353d5f639 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:59.627888 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e938d4b-8a9b-4a2f-ae60-a10353d5f639 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:59.628300 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e938d4b-8a9b-4a2f-ae60-a10353d5f639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:59.649312 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738a8cef-1b01-45ed-ae26-5fb0dff2b630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:59.649968 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738a8cef-1b01-45ed-ae26-5fb0dff2b630 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:59.650243 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738a8cef-1b01-45ed-ae26-5fb0dff2b630 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:59.650515 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738a8cef-1b01-45ed-ae26-5fb0dff2b630 None admin] PATH_INFO: `/v3/projects/f8b7cfbd340b4753bd898ddc0a462a03` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:59.652318 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738a8cef-1b01-45ed-ae26-5fb0dff2b630 None admin] RBAC: Authorizing `identity:get_project(project_id=f8b7cfbd340b4753bd898ddc0a462a03)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:59.653811 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738a8cef-1b01-45ed-ae26-5fb0dff2b630 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:59.655614 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 172/346] 10.222.0.55 () {56 vars in 1186 bytes} [Wed Aug 24 08:33:59 2022] GET /identity/v3/projects/f8b7cfbd340b4753bd898ddc0a462a03 => generated 298 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:33:59.656300 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738a8cef-1b01-45ed-ae26-5fb0dff2b630 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:59.657533 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-4e938d4b-8a9b-4a2f-ae60-a10353d5f639 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:33:59.658740 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-4e938d4b-8a9b-4a2f-ae60-a10353d5f639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:33:59.773982 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:59.786824 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738a8cef-1b01-45ed-ae26-5fb0dff2b630 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:59.808633 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 175/347] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:33:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:33:59.808633 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e938d4b-8a9b-4a2f-ae60-a10353d5f639 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:59.834630 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1e48a6-2023-44c0-8eeb-1a5671d45df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:33:59.835547 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1e48a6-2023-44c0-8eeb-1a5671d45df3 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:33:59.836006 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1e48a6-2023-44c0-8eeb-1a5671d45df3 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:33:59.836481 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1e48a6-2023-44c0-8eeb-1a5671d45df3 None admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:33:59.837529 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1e48a6-2023-44c0-8eeb-1a5671d45df3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:33:59.838009 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1e48a6-2023-44c0-8eeb-1a5671d45df3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:33:59.840327 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1e48a6-2023-44c0-8eeb-1a5671d45df3 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:33:59.854383 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 173/348] 10.222.0.55 () {56 vars in 1111 bytes} [Wed Aug 24 08:33:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:33:59.860865 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1e48a6-2023-44c0-8eeb-1a5671d45df3 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:33:59.905056 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:33:59.963104 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e938d4b-8a9b-4a2f-ae60-a10353d5f639 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:33:59.993406 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1e48a6-2023-44c0-8eeb-1a5671d45df3 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:00.035908 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f671d789-3010-42e8-94b3-3cbd7e79873e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:00.036626 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f671d789-3010-42e8-94b3-3cbd7e79873e None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:00.036957 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f671d789-3010-42e8-94b3-3cbd7e79873e None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:00.037240 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f671d789-3010-42e8-94b3-3cbd7e79873e None admin] PATH_INFO: `/v3/groups/121aa5cb19db48029d2846f8b2bf90de` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:00.039415 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c93fa85-03a5-45e0-ab11-dc8a6dfe1978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:00.040744 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c93fa85-03a5-45e0-ab11-dc8a6dfe1978 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:00.041536 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c93fa85-03a5-45e0-ab11-dc8a6dfe1978 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:00.042162 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c93fa85-03a5-45e0-ab11-dc8a6dfe1978 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:00.053360 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f671d789-3010-42e8-94b3-3cbd7e79873e None admin] RBAC: Authorizing `identity:get_group(group_id=121aa5cb19db48029d2846f8b2bf90de)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:00.054844 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f671d789-3010-42e8-94b3-3cbd7e79873e None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:00.056595 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 176/349] 10.222.0.55 () {56 vars in 1180 bytes} [Wed Aug 24 08:33:59 2022] GET /identity/v3/groups/121aa5cb19db48029d2846f8b2bf90de => generated 226 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:00.057943 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f671d789-3010-42e8-94b3-3cbd7e79873e None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:00.125815 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:00.129187 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c93fa85-03a5-45e0-ab11-dc8a6dfe1978 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:00.130994 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c93fa85-03a5-45e0-ab11-dc8a6dfe1978 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:00.167006 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:00.180150 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f671d789-3010-42e8-94b3-3cbd7e79873e None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:00.232359 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cde1518-fa1d-46f6-ba6c-89e149e2fe1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:00.232359 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cde1518-fa1d-46f6-ba6c-89e149e2fe1a None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:00.232359 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cde1518-fa1d-46f6-ba6c-89e149e2fe1a None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:00.232359 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cde1518-fa1d-46f6-ba6c-89e149e2fe1a None admin] PATH_INFO: `/v3/roles/member` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:00.247100 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cde1518-fa1d-46f6-ba6c-89e149e2fe1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:34:00.252795 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-9cde1518-fa1d-46f6-ba6c-89e149e2fe1a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:34:00.252795 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:00.252795 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:00.252795 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:00.252795 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:00.252795 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:00.252795 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:00.252795 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:00.252795 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:00.252795 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:00.252795 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:00.252795 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:00.252795 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:34:00.252795 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:34:00.252795 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:34:00.252795 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:34:00.254384 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:34:00.254384 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:34:00.254384 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:34:00.254384 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:34:00.254384 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:00.254384 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:00.254384 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:00.254384 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:00.254384 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:00.254384 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:00.254384 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:00.254384 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:00.254384 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:00.254384 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:00.254384 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:00.254384 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:00.255482 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:00.255482 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:00.255482 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:00.255482 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:00.255482 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:34:00.255482 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:34:00.255482 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:34:00.255482 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:34:00.255482 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:34:00.255482 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:34:00.255482 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:34:00.255482 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:34:00.255482 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 177/350] 10.222.0.55 () {56 vars in 1099 bytes} [Wed Aug 24 08:34:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:00.261010 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aec9218-2360-4c6a-bf1c-2baee2ec40cf None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:00.261571 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aec9218-2360-4c6a-bf1c-2baee2ec40cf None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:00.262325 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aec9218-2360-4c6a-bf1c-2baee2ec40cf None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:00.264313 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 178/351] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:34:00 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:34:00.264906 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aec9218-2360-4c6a-bf1c-2baee2ec40cf None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:00.293510 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 174/352] 10.222.0.55 () {58 vars in 1282 bytes} [Wed Aug 24 08:33:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 08:34:00.294186 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c93fa85-03a5-45e0-ab11-dc8a6dfe1978 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:00.413061 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aec9218-2360-4c6a-bf1c-2baee2ec40cf None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:00.427702 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c93fa85-03a5-45e0-ab11-dc8a6dfe1978 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:00.428806 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:00.461757 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:00.468375 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f2b07a-eae0-4a5e-806d-baa21b4f1b7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:00.468375 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f2b07a-eae0-4a5e-806d-baa21b4f1b7b None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:00.468649 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f2b07a-eae0-4a5e-806d-baa21b4f1b7b None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:00.468831 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f2b07a-eae0-4a5e-806d-baa21b4f1b7b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:00.469755 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f2b07a-eae0-4a5e-806d-baa21b4f1b7b None admin] RBAC: Adding query filter params (group.id=121aa5cb19db48029d2846f8b2bf90de, role.id=b674fd0a592e4e0cada811bba5ceea4d, scope.project.id=f8b7cfbd340b4753bd898ddc0a462a03) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:00.469904 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f2b07a-eae0-4a5e-806d-baa21b4f1b7b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:00.471704 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f2b07a-eae0-4a5e-806d-baa21b4f1b7b None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:00.478867 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c790b53-0477-403b-98ff-b1f3157e111b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:00.479630 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 179/353] 10.222.0.55 () {56 vars in 1376 bytes} [Wed Aug 24 08:34:00 2022] GET /identity/v3/role_assignments?group.id=121aa5cb19db48029d2846f8b2bf90de&scope.project.id=f8b7cfbd340b4753bd898ddc0a462a03&role.id=b674fd0a592e4e0cada811bba5ceea4d => generated 260 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:00.479791 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c790b53-0477-403b-98ff-b1f3157e111b None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:00.479917 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f2b07a-eae0-4a5e-806d-baa21b4f1b7b None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:00.480041 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c790b53-0477-403b-98ff-b1f3157e111b None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:00.480447 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c790b53-0477-403b-98ff-b1f3157e111b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:00.507861 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c790b53-0477-403b-98ff-b1f3157e111b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:34:00.517099 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-7c790b53-0477-403b-98ff-b1f3157e111b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 08:34:00.517099 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:00.517099 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:00.517099 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:00.517099 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:00.517099 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:00.517099 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:00.517099 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:00.517099 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:00.517099 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:00.517099 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:00.517099 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:00.517099 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:34:00.517099 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:34:00.517099 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:34:00.517099 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:34:00.518884 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:34:00.518884 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:34:00.518884 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:34:00.518884 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:34:00.518884 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:00.518884 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:00.518884 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:00.518884 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:00.518884 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:00.518884 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:00.518884 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:00.518884 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:00.518884 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:00.518884 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:00.518884 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:00.518884 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:00.523351 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:00.523351 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:00.523351 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:00.523351 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:00.523351 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:34:00.523351 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:34:00.523351 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:34:00.523351 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:34:00.523351 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:34:00.523351 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:34:00.523351 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 08:34:00.523351 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:34:00.523351 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 175/354] 10.222.0.55 () {56 vars in 1099 bytes} [Wed Aug 24 08:34:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:00.523351 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488dd897-71ba-4e66-8dbd-ba2318af4c28 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:00.525537 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488dd897-71ba-4e66-8dbd-ba2318af4c28 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:00.525537 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488dd897-71ba-4e66-8dbd-ba2318af4c28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:00.548309 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-488dd897-71ba-4e66-8dbd-ba2318af4c28 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:00.549684 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-488dd897-71ba-4e66-8dbd-ba2318af4c28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:00.643567 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f2b07a-eae0-4a5e-806d-baa21b4f1b7b None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:00.650586 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 176/355] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:34:00.651783 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488dd897-71ba-4e66-8dbd-ba2318af4c28 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:00.694323 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5f2df2-3f41-4bba-bceb-d7aaa0b43265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:00.695217 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5f2df2-3f41-4bba-bceb-d7aaa0b43265 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:00.695505 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5f2df2-3f41-4bba-bceb-d7aaa0b43265 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:00.695785 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5f2df2-3f41-4bba-bceb-d7aaa0b43265 None admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:00.696476 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5f2df2-3f41-4bba-bceb-d7aaa0b43265 None admin] RBAC: Adding query filter params (name=member) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:00.696840 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5f2df2-3f41-4bba-bceb-d7aaa0b43265 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:00.699349 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5f2df2-3f41-4bba-bceb-d7aaa0b43265 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:00.710544 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 180/356] 10.222.0.55 () {56 vars in 1101 bytes} [Wed Aug 24 08:34:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:00.711849 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036317b4-64d1-4c8f-9b27-03fd27c3524f None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:00.712208 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036317b4-64d1-4c8f-9b27-03fd27c3524f None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:00.712506 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036317b4-64d1-4c8f-9b27-03fd27c3524f None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:00.714672 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 181/357] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:34:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:34:00.715432 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036317b4-64d1-4c8f-9b27-03fd27c3524f None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:00.763230 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:00.794721 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488dd897-71ba-4e66-8dbd-ba2318af4c28 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:00.820027 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:00.845282 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca75d2c0-adcb-4e22-9dfb-750b680d440d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:00.846242 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca75d2c0-adcb-4e22-9dfb-750b680d440d None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:00.846936 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca75d2c0-adcb-4e22-9dfb-750b680d440d None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:00.847848 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca75d2c0-adcb-4e22-9dfb-750b680d440d None admin] PATH_INFO: `/v3/projects/f8b7cfbd340b4753bd898ddc0a462a03` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:00.849269 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca75d2c0-adcb-4e22-9dfb-750b680d440d None admin] RBAC: Authorizing `identity:get_project(project_id=f8b7cfbd340b4753bd898ddc0a462a03)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:00.851532 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca75d2c0-adcb-4e22-9dfb-750b680d440d None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:00.855288 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 177/358] 10.222.0.55 () {56 vars in 1186 bytes} [Wed Aug 24 08:34:00 2022] GET /identity/v3/projects/f8b7cfbd340b4753bd898ddc0a462a03 => generated 298 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:00.857832 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac6db97-707e-44cf-8551-ece57a3629e1 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:00.858495 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac6db97-707e-44cf-8551-ece57a3629e1 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:00.859085 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac6db97-707e-44cf-8551-ece57a3629e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:00.871274 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036317b4-64d1-4c8f-9b27-03fd27c3524f None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:00.882690 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-3ac6db97-707e-44cf-8551-ece57a3629e1 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:00.884804 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-3ac6db97-707e-44cf-8551-ece57a3629e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:00.929405 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73be5158-774d-49cc-a0d9-017deccc1f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:00.931217 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73be5158-774d-49cc-a0d9-017deccc1f9b None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:00.931503 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73be5158-774d-49cc-a0d9-017deccc1f9b None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:00.931784 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73be5158-774d-49cc-a0d9-017deccc1f9b None admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:00.932511 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73be5158-774d-49cc-a0d9-017deccc1f9b None admin] RBAC: Adding query filter params (name=reader) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:00.932842 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73be5158-774d-49cc-a0d9-017deccc1f9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:00.934526 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73be5158-774d-49cc-a0d9-017deccc1f9b None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:00.951152 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 182/359] 10.222.0.55 () {56 vars in 1101 bytes} [Wed Aug 24 08:34:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:00.951152 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73be5158-774d-49cc-a0d9-017deccc1f9b None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:01.012203 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 178/360] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:34:01.014685 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac6db97-707e-44cf-8551-ece57a3629e1 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:01.105413 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:01.110944 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73be5158-774d-49cc-a0d9-017deccc1f9b None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:01.165366 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bb553e-9430-458f-8168-5f6b467462da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:01.166362 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bb553e-9430-458f-8168-5f6b467462da None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:01.166933 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bb553e-9430-458f-8168-5f6b467462da None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:01.167372 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bb553e-9430-458f-8168-5f6b467462da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:01.199739 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac6db97-707e-44cf-8551-ece57a3629e1 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:01.224529 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:01.245400 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bb553e-9430-458f-8168-5f6b467462da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:01.246951 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bb553e-9430-458f-8168-5f6b467462da None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:01.259945 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf64a1b-9ebd-4b56-ad27-c23c05ce29d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:01.261854 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf64a1b-9ebd-4b56-ad27-c23c05ce29d8 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:01.262528 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf64a1b-9ebd-4b56-ad27-c23c05ce29d8 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:01.263281 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf64a1b-9ebd-4b56-ad27-c23c05ce29d8 None admin] PATH_INFO: `/v3/groups/121aa5cb19db48029d2846f8b2bf90de` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:01.265619 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf64a1b-9ebd-4b56-ad27-c23c05ce29d8 None admin] RBAC: Authorizing `identity:get_group(group_id=121aa5cb19db48029d2846f8b2bf90de)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:01.268750 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf64a1b-9ebd-4b56-ad27-c23c05ce29d8 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:01.272116 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 179/361] 10.222.0.55 () {56 vars in 1180 bytes} [Wed Aug 24 08:34:01 2022] GET /identity/v3/groups/121aa5cb19db48029d2846f8b2bf90de => generated 226 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:01.273860 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf64a1b-9ebd-4b56-ad27-c23c05ce29d8 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:01.361479 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:01.367134 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 183/362] 10.222.0.55 () {58 vars in 1282 bytes} [Wed Aug 24 08:34:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:34:01.367997 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bb553e-9430-458f-8168-5f6b467462da None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:01.491413 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf64a1b-9ebd-4b56-ad27-c23c05ce29d8 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:01.521380 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bb553e-9430-458f-8168-5f6b467462da None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:01.563223 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1622aa1d-e716-4347-a1a4-5b73d3a29d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:01.564646 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1622aa1d-e716-4347-a1a4-5b73d3a29d63 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:01.565390 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1622aa1d-e716-4347-a1a4-5b73d3a29d63 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:01.566082 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1622aa1d-e716-4347-a1a4-5b73d3a29d63 None admin] PATH_INFO: `/v3/users/73ee7493ec2f4d019cd87308b7d19514` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:01.568508 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1622aa1d-e716-4347-a1a4-5b73d3a29d63 None admin] RBAC: Authorizing `identity:get_user(user_id=73ee7493ec2f4d019cd87308b7d19514)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:01.570860 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1622aa1d-e716-4347-a1a4-5b73d3a29d63 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:01.572515 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 180/363] 10.222.0.55 () {56 vars in 1177 bytes} [Wed Aug 24 08:34:01 2022] GET /identity/v3/users/73ee7493ec2f4d019cd87308b7d19514 => generated 293 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:01.573782 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1622aa1d-e716-4347-a1a4-5b73d3a29d63 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:01.577262 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906a039a-d457-424a-93b5-6c5906bf4379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:01.578222 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906a039a-d457-424a-93b5-6c5906bf4379 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:01.578627 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906a039a-d457-424a-93b5-6c5906bf4379 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:01.579023 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906a039a-d457-424a-93b5-6c5906bf4379 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:01.602279 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906a039a-d457-424a-93b5-6c5906bf4379 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:34:01.614137 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-906a039a-d457-424a-93b5-6c5906bf4379 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 08:34:01.614137 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:01.614137 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:01.614137 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:01.614137 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:01.614137 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:01.614137 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:01.614137 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:01.614137 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:01.614137 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:01.614137 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:01.614137 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:01.614137 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:34:01.614137 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:34:01.614137 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:34:01.614137 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:34:01.615913 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:34:01.615913 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:34:01.615913 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:34:01.615913 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:34:01.615913 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:01.615913 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:01.615913 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:01.615913 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:01.615913 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:01.615913 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:01.615913 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:01.615913 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:01.615913 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:01.615913 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:01.615913 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:01.615913 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:01.618007 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:01.618007 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:01.618007 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:01.618007 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:01.618007 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:34:01.618007 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:34:01.618007 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:34:01.618007 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:34:01.618007 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:34:01.618007 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:34:01.618007 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 08:34:01.618007 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:34:01.618007 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 184/364] 10.222.0.55 () {56 vars in 1114 bytes} [Wed Aug 24 08:34:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:01.619312 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906a039a-d457-424a-93b5-6c5906bf4379 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:01.651812 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:01.710685 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1622aa1d-e716-4347-a1a4-5b73d3a29d63 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:01.737819 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:01.776477 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3592f6b7-15db-4719-8a40-4a4de73f4e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:01.777419 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3592f6b7-15db-4719-8a40-4a4de73f4e94 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:01.777883 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3592f6b7-15db-4719-8a40-4a4de73f4e94 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:01.778342 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3592f6b7-15db-4719-8a40-4a4de73f4e94 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:01.779640 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3592f6b7-15db-4719-8a40-4a4de73f4e94 None admin] RBAC: Adding query filter params (group.id=121aa5cb19db48029d2846f8b2bf90de, role.id=8b5ad96041894155b0138a676ead607e, scope.project.id=f8b7cfbd340b4753bd898ddc0a462a03) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:01.780128 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3592f6b7-15db-4719-8a40-4a4de73f4e94 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:01.782303 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3592f6b7-15db-4719-8a40-4a4de73f4e94 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:01.791368 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 181/365] 10.222.0.55 () {56 vars in 1376 bytes} [Wed Aug 24 08:34:01 2022] GET /identity/v3/role_assignments?group.id=121aa5cb19db48029d2846f8b2bf90de&scope.project.id=f8b7cfbd340b4753bd898ddc0a462a03&role.id=8b5ad96041894155b0138a676ead607e => generated 260 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:01.792875 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3592f6b7-15db-4719-8a40-4a4de73f4e94 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:01.809200 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906a039a-d457-424a-93b5-6c5906bf4379 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:01.861588 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48306588-2822-4707-b2d0-bd579b1f1cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:01.862556 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48306588-2822-4707-b2d0-bd579b1f1cad None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:01.863063 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48306588-2822-4707-b2d0-bd579b1f1cad None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:01.863512 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48306588-2822-4707-b2d0-bd579b1f1cad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:01.877060 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48306588-2822-4707-b2d0-bd579b1f1cad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:34:01.885166 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-48306588-2822-4707-b2d0-bd579b1f1cad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:34:01.885166 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:01.885166 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:01.885166 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:01.885166 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:01.885166 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:01.885166 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:01.885166 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:01.885166 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:01.885166 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:01.885166 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:01.885166 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:01.885166 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:34:01.885166 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:34:01.885166 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:34:01.885166 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:34:01.886181 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:34:01.886181 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:34:01.886181 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:34:01.886181 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:34:01.886181 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:01.886181 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:01.886181 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:01.886181 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:01.886181 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:01.886181 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:01.886181 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:01.886181 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:01.886181 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:01.886181 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:01.886181 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:01.886181 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:01.886971 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:01.886971 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:01.886971 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:01.886971 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:01.886971 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:34:01.886971 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:34:01.886971 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:34:01.886971 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:34:01.886971 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:34:01.886971 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:34:01.886971 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:34:01.886971 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:34:01.889200 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 185/366] 10.222.0.55 () {56 vars in 1096 bytes} [Wed Aug 24 08:34:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:01.890226 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48306588-2822-4707-b2d0-bd579b1f1cad None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:01.972704 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3592f6b7-15db-4719-8a40-4a4de73f4e94 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:01.992961 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:02.049935 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03a476f-cd94-4f87-a0f9-850af3a61f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:02.050487 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03a476f-cd94-4f87-a0f9-850af3a61f16 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:02.050764 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03a476f-cd94-4f87-a0f9-850af3a61f16 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:02.051037 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03a476f-cd94-4f87-a0f9-850af3a61f16 None admin] PATH_INFO: `/v3/projects/61247a1049b846b79052f42b0d525259` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:02.055612 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03a476f-cd94-4f87-a0f9-850af3a61f16 None admin] RBAC: Authorizing `identity:get_project(project_id=61247a1049b846b79052f42b0d525259)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:02.056728 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03a476f-cd94-4f87-a0f9-850af3a61f16 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:02.059545 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 182/367] 10.222.0.55 () {56 vars in 1186 bytes} [Wed Aug 24 08:34:01 2022] GET /identity/v3/projects/61247a1049b846b79052f42b0d525259 => generated 302 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:02.060360 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03a476f-cd94-4f87-a0f9-850af3a61f16 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:02.081047 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48306588-2822-4707-b2d0-bd579b1f1cad None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:02.114715 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:02.147408 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6902522-5236-490d-9833-f2553bd7f84d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:02.147895 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6902522-5236-490d-9833-f2553bd7f84d None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:02.148952 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6902522-5236-490d-9833-f2553bd7f84d None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:02.149522 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6902522-5236-490d-9833-f2553bd7f84d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:02.151291 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6902522-5236-490d-9833-f2553bd7f84d None admin] RBAC: Adding query filter params (role.id=e308031ba9a84b79bcdf501455ebe0d7, user.id=73ee7493ec2f4d019cd87308b7d19514, scope.system=all) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:02.152021 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6902522-5236-490d-9833-f2553bd7f84d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:02.155415 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6902522-5236-490d-9833-f2553bd7f84d None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:02.183150 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 186/368] 10.222.0.55 () {56 vars in 1308 bytes} [Wed Aug 24 08:34:01 2022] GET /identity/v3/role_assignments?user.id=73ee7493ec2f4d019cd87308b7d19514&scope.system=all&role.id=e308031ba9a84b79bcdf501455ebe0d7 => generated 510 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:02.188904 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6902522-5236-490d-9833-f2553bd7f84d None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:02.193025 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03a476f-cd94-4f87-a0f9-850af3a61f16 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:02.245135 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1853566f-9e97-45f1-aba1-d48095694b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:02.246121 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1853566f-9e97-45f1-aba1-d48095694b28 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:02.246121 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1853566f-9e97-45f1-aba1-d48095694b28 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:02.246364 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1853566f-9e97-45f1-aba1-d48095694b28 None admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:02.247474 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1853566f-9e97-45f1-aba1-d48095694b28 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:02.247971 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1853566f-9e97-45f1-aba1-d48095694b28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:02.249750 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1853566f-9e97-45f1-aba1-d48095694b28 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:02.260841 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 183/369] 10.222.0.55 () {56 vars in 1111 bytes} [Wed Aug 24 08:34:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:02.261788 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1853566f-9e97-45f1-aba1-d48095694b28 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:02.378051 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:02.402763 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6902522-5236-490d-9833-f2553bd7f84d None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:02.449654 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a4582b-5c72-4dab-aa09-583888a15cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:02.450726 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a4582b-5c72-4dab-aa09-583888a15cfb None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:02.451206 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a4582b-5c72-4dab-aa09-583888a15cfb None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:02.452415 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a4582b-5c72-4dab-aa09-583888a15cfb None admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:02.453389 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a4582b-5c72-4dab-aa09-583888a15cfb None admin] RBAC: Adding query filter params (name=admin) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:02.453925 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a4582b-5c72-4dab-aa09-583888a15cfb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:02.455519 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a4582b-5c72-4dab-aa09-583888a15cfb None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:02.471055 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 187/370] 10.222.0.55 () {56 vars in 1099 bytes} [Wed Aug 24 08:34:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:02.472276 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a4582b-5c72-4dab-aa09-583888a15cfb None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:02.523234 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1853566f-9e97-45f1-aba1-d48095694b28 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:02.531256 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:02.594816 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b552678b-939a-466e-8933-f49adb2ac020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:02.595811 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b552678b-939a-466e-8933-f49adb2ac020 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:02.596251 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b552678b-939a-466e-8933-f49adb2ac020 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:02.596736 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b552678b-939a-466e-8933-f49adb2ac020 None admin] PATH_INFO: `/v3/groups/121aa5cb19db48029d2846f8b2bf90de` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:02.598752 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b552678b-939a-466e-8933-f49adb2ac020 None admin] RBAC: Authorizing `identity:get_group(group_id=121aa5cb19db48029d2846f8b2bf90de)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:02.601447 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b552678b-939a-466e-8933-f49adb2ac020 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:02.604919 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 184/371] 10.222.0.55 () {56 vars in 1180 bytes} [Wed Aug 24 08:34:02 2022] GET /identity/v3/groups/121aa5cb19db48029d2846f8b2bf90de => generated 226 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:02.606314 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b552678b-939a-466e-8933-f49adb2ac020 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:02.656834 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a4582b-5c72-4dab-aa09-583888a15cfb None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:02.726254 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041f83d5-368b-4d7f-98c7-b3e11713afed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:02.726955 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041f83d5-368b-4d7f-98c7-b3e11713afed None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:02.727239 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041f83d5-368b-4d7f-98c7-b3e11713afed None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:02.727501 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041f83d5-368b-4d7f-98c7-b3e11713afed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:02.767598 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b552678b-939a-466e-8933-f49adb2ac020 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:02.800976 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:02.834751 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d4dd81-bf3a-4a83-9579-1ab5f4f0da00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:02.835720 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d4dd81-bf3a-4a83-9579-1ab5f4f0da00 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:02.836164 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d4dd81-bf3a-4a83-9579-1ab5f4f0da00 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:02.836659 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d4dd81-bf3a-4a83-9579-1ab5f4f0da00 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:02.846884 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041f83d5-368b-4d7f-98c7-b3e11713afed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:02.848497 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041f83d5-368b-4d7f-98c7-b3e11713afed None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:02.906057 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:02.951290 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d4dd81-bf3a-4a83-9579-1ab5f4f0da00 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:02.951940 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d4dd81-bf3a-4a83-9579-1ab5f4f0da00 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:03.001261 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 188/372] 10.222.0.55 () {58 vars in 1282 bytes} [Wed Aug 24 08:34:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 529 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:34:03.003262 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041f83d5-368b-4d7f-98c7-b3e11713afed None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:03.119576 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 185/373] 10.222.0.55 () {58 vars in 1282 bytes} [Wed Aug 24 08:34:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 514 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:34:03.123400 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4472b2a-1ba0-4252-a4f4-eb14f4f0f02d None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:03.124137 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4472b2a-1ba0-4252-a4f4-eb14f4f0f02d None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:03.125201 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4472b2a-1ba0-4252-a4f4-eb14f4f0f02d None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:03.128125 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 186/374] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:34:03 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:34:03.129722 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4472b2a-1ba0-4252-a4f4-eb14f4f0f02d None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:03.152841 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:03.169988 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041f83d5-368b-4d7f-98c7-b3e11713afed None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:03.219688 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116532ff-037e-443a-bc6c-d70da6c0a8be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:03.220240 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116532ff-037e-443a-bc6c-d70da6c0a8be None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:03.220518 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116532ff-037e-443a-bc6c-d70da6c0a8be None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:03.220825 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116532ff-037e-443a-bc6c-d70da6c0a8be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:03.221596 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116532ff-037e-443a-bc6c-d70da6c0a8be None admin] RBAC: Adding query filter params (group.id=121aa5cb19db48029d2846f8b2bf90de, role.id=b674fd0a592e4e0cada811bba5ceea4d, scope.project.id=61247a1049b846b79052f42b0d525259) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:03.221901 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116532ff-037e-443a-bc6c-d70da6c0a8be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:03.225858 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116532ff-037e-443a-bc6c-d70da6c0a8be None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:03.233896 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 189/375] 10.222.0.55 () {56 vars in 1376 bytes} [Wed Aug 24 08:34:03 2022] GET /identity/v3/role_assignments?group.id=121aa5cb19db48029d2846f8b2bf90de&scope.project.id=61247a1049b846b79052f42b0d525259&role.id=b674fd0a592e4e0cada811bba5ceea4d => generated 260 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:03.234673 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116532ff-037e-443a-bc6c-d70da6c0a8be None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:03.240397 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:03.304671 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4472b2a-1ba0-4252-a4f4-eb14f4f0f02d None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:03.369216 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03be100-d781-44e5-b30c-18c6a1389381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:03.371033 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03be100-d781-44e5-b30c-18c6a1389381 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:03.371033 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03be100-d781-44e5-b30c-18c6a1389381 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:03.371272 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03be100-d781-44e5-b30c-18c6a1389381 None admin] PATH_INFO: `/v3/projects/61247a1049b846b79052f42b0d525259` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:03.372677 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03be100-d781-44e5-b30c-18c6a1389381 None admin] RBAC: Authorizing `identity:get_project(project_id=61247a1049b846b79052f42b0d525259)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:03.374265 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03be100-d781-44e5-b30c-18c6a1389381 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:03.376700 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 187/376] 10.222.0.55 () {56 vars in 1186 bytes} [Wed Aug 24 08:34:03 2022] GET /identity/v3/projects/61247a1049b846b79052f42b0d525259 => generated 302 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:03.377923 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43b65a5-f301-4c61-a4f4-23fa6326a940 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:03.378275 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43b65a5-f301-4c61-a4f4-23fa6326a940 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:03.378616 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43b65a5-f301-4c61-a4f4-23fa6326a940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:03.408240 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-e43b65a5-f301-4c61-a4f4-23fa6326a940 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:03.410602 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-e43b65a5-f301-4c61-a4f4-23fa6326a940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:03.412527 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116532ff-037e-443a-bc6c-d70da6c0a8be None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:03.472377 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede795a7-5c0a-433a-af3e-1cb8b8521418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:03.473222 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede795a7-5c0a-433a-af3e-1cb8b8521418 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:03.473584 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede795a7-5c0a-433a-af3e-1cb8b8521418 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:03.473922 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede795a7-5c0a-433a-af3e-1cb8b8521418 None admin] PATH_INFO: `/v3/projects/6ad87fa1813247aba83e93f886f34eb3` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:03.476489 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede795a7-5c0a-433a-af3e-1cb8b8521418 None admin] RBAC: Authorizing `identity:get_project(project_id=6ad87fa1813247aba83e93f886f34eb3)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:03.477112 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede795a7-5c0a-433a-af3e-1cb8b8521418 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:03.479975 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 190/377] 10.222.0.55 () {56 vars in 1186 bytes} [Wed Aug 24 08:34:03 2022] GET /identity/v3/projects/6ad87fa1813247aba83e93f886f34eb3 => generated 344 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:03.484870 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede795a7-5c0a-433a-af3e-1cb8b8521418 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:03.546325 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 188/378] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:34:03.548381 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43b65a5-f301-4c61-a4f4-23fa6326a940 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:03.571430 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:03.591088 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:03.639093 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede795a7-5c0a-433a-af3e-1cb8b8521418 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:03.684060 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f362b3-cd79-4202-a25f-3747eb5e238c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:03.684742 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f362b3-cd79-4202-a25f-3747eb5e238c None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:03.685778 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f362b3-cd79-4202-a25f-3747eb5e238c None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:03.685778 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f362b3-cd79-4202-a25f-3747eb5e238c None admin] PATH_INFO: `/v3/groups/121aa5cb19db48029d2846f8b2bf90de` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:03.687869 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f362b3-cd79-4202-a25f-3747eb5e238c None admin] RBAC: Authorizing `identity:get_group(group_id=121aa5cb19db48029d2846f8b2bf90de)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:03.689350 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f362b3-cd79-4202-a25f-3747eb5e238c None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:03.694187 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 191/379] 10.222.0.55 () {56 vars in 1180 bytes} [Wed Aug 24 08:34:03 2022] GET /identity/v3/groups/121aa5cb19db48029d2846f8b2bf90de => generated 226 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:03.698572 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f362b3-cd79-4202-a25f-3747eb5e238c None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:03.774778 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43b65a5-f301-4c61-a4f4-23fa6326a940 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:03.850240 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b1a432-81a7-4f1d-9796-6cf79567e043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:03.851794 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b1a432-81a7-4f1d-9796-6cf79567e043 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:03.852559 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b1a432-81a7-4f1d-9796-6cf79567e043 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:03.853308 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b1a432-81a7-4f1d-9796-6cf79567e043 None admin] PATH_INFO: `/v3/groups/a37059af981c4b8b8bee1a21f0f63ebb` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:03.866316 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f362b3-cd79-4202-a25f-3747eb5e238c None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:03.867331 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b1a432-81a7-4f1d-9796-6cf79567e043 None admin] RBAC: Authorizing `identity:get_group(group_id=a37059af981c4b8b8bee1a21f0f63ebb)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:03.870338 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b1a432-81a7-4f1d-9796-6cf79567e043 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:03.874050 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 189/380] 10.222.0.55 () {56 vars in 1180 bytes} [Wed Aug 24 08:34:03 2022] GET /identity/v3/groups/a37059af981c4b8b8bee1a21f0f63ebb => generated 229 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:03.883947 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b1a432-81a7-4f1d-9796-6cf79567e043 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:03.913509 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccefbeae-334f-4f74-961d-daed1efd0782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:03.914134 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccefbeae-334f-4f74-961d-daed1efd0782 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:03.914411 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccefbeae-334f-4f74-961d-daed1efd0782 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:03.914681 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccefbeae-334f-4f74-961d-daed1efd0782 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:03.915454 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccefbeae-334f-4f74-961d-daed1efd0782 None admin] RBAC: Adding query filter params (group.id=121aa5cb19db48029d2846f8b2bf90de, role.id=8b5ad96041894155b0138a676ead607e, scope.project.id=61247a1049b846b79052f42b0d525259) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:03.915765 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccefbeae-334f-4f74-961d-daed1efd0782 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:03.917919 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccefbeae-334f-4f74-961d-daed1efd0782 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:03.924965 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 192/381] 10.222.0.55 () {56 vars in 1376 bytes} [Wed Aug 24 08:34:03 2022] GET /identity/v3/role_assignments?group.id=121aa5cb19db48029d2846f8b2bf90de&scope.project.id=61247a1049b846b79052f42b0d525259&role.id=8b5ad96041894155b0138a676ead607e => generated 260 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:04.035307 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:04.060986 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b1a432-81a7-4f1d-9796-6cf79567e043 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:04.128713 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be84cc8a-78fc-46bf-9077-bfa8ca1b02c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:04.129703 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be84cc8a-78fc-46bf-9077-bfa8ca1b02c3 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:04.130120 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be84cc8a-78fc-46bf-9077-bfa8ca1b02c3 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:04.130563 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be84cc8a-78fc-46bf-9077-bfa8ca1b02c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:04.131935 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be84cc8a-78fc-46bf-9077-bfa8ca1b02c3 None admin] RBAC: Adding query filter params (group.id=a37059af981c4b8b8bee1a21f0f63ebb, role.id=b870cd1acaa744e1ae1d58b9ac283821, scope.project.id=6ad87fa1813247aba83e93f886f34eb3) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:04.132490 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be84cc8a-78fc-46bf-9077-bfa8ca1b02c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:04.135312 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be84cc8a-78fc-46bf-9077-bfa8ca1b02c3 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:04.147512 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 190/382] 10.222.0.55 () {56 vars in 1376 bytes} [Wed Aug 24 08:34:03 2022] GET /identity/v3/role_assignments?group.id=a37059af981c4b8b8bee1a21f0f63ebb&scope.project.id=6ad87fa1813247aba83e93f886f34eb3&role.id=b870cd1acaa744e1ae1d58b9ac283821 => generated 260 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:34:04.255360 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccefbeae-334f-4f74-961d-daed1efd0782 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:04.259869 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:04.447706 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccefbeae-334f-4f74-961d-daed1efd0782 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:04.502417 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafbc5f1-aaa0-43da-a1cf-f24add8babb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:04.503460 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafbc5f1-aaa0-43da-a1cf-f24add8babb5 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:04.503968 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafbc5f1-aaa0-43da-a1cf-f24add8babb5 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:04.504450 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafbc5f1-aaa0-43da-a1cf-f24add8babb5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:04.526559 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafbc5f1-aaa0-43da-a1cf-f24add8babb5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:34:04.533171 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-fafbc5f1-aaa0-43da-a1cf-f24add8babb5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:34:04.533171 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:04.533171 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:04.533171 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:04.533171 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:04.533171 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:04.533171 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:04.533171 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:04.533171 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:04.533171 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:04.533171 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:04.533171 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:04.533171 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:34:04.533171 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:34:04.533171 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:34:04.533171 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:34:04.534153 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:34:04.534153 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:34:04.534153 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:34:04.534153 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:34:04.534153 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:04.534153 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:04.534153 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:04.534153 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:04.534153 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:04.534153 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:04.534153 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:04.534153 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:04.534153 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:04.534153 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:04.534153 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:04.534153 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:04.535033 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:04.535033 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:04.535033 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:04.535033 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:04.535033 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:34:04.535033 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:34:04.535033 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:34:04.535033 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:34:04.535033 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:34:04.535033 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:34:04.535033 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:34:04.535033 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:34:04.537446 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 193/383] 10.222.0.55 () {56 vars in 1099 bytes} [Wed Aug 24 08:34:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:04.547132 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be84cc8a-78fc-46bf-9077-bfa8ca1b02c3 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:04.823848 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be84cc8a-78fc-46bf-9077-bfa8ca1b02c3 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:04.881017 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1855a1-2f5b-4bb7-b528-c7c4c0acc740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:04.882035 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1855a1-2f5b-4bb7-b528-c7c4c0acc740 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:04.882551 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1855a1-2f5b-4bb7-b528-c7c4c0acc740 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:04.883032 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1855a1-2f5b-4bb7-b528-c7c4c0acc740 None admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:04.883916 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1855a1-2f5b-4bb7-b528-c7c4c0acc740 None admin] RBAC: Adding query filter params (name=member) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:04.884431 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1855a1-2f5b-4bb7-b528-c7c4c0acc740 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:04.886111 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1855a1-2f5b-4bb7-b528-c7c4c0acc740 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:04.897641 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:04.902258 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 191/384] 10.222.0.55 () {56 vars in 1101 bytes} [Wed Aug 24 08:34:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:04.910443 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafbc5f1-aaa0-43da-a1cf-f24add8babb5 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:05.023461 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77efd120-6104-4767-bc24-b2b11e59bce5 None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:05.023741 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77efd120-6104-4767-bc24-b2b11e59bce5 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:05.024019 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77efd120-6104-4767-bc24-b2b11e59bce5 None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:05.025688 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 192/385] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:34:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:34:05.046990 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8ff60b-505b-4cc9-98e0-01ff471fbb20 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:05.048045 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8ff60b-505b-4cc9-98e0-01ff471fbb20 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:05.050939 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8ff60b-505b-4cc9-98e0-01ff471fbb20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:05.085068 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:05.093651 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-7d8ff60b-505b-4cc9-98e0-01ff471fbb20 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:05.099065 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-7d8ff60b-505b-4cc9-98e0-01ff471fbb20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:05.183625 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafbc5f1-aaa0-43da-a1cf-f24add8babb5 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:05.292530 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 193/386] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 08:34:05.302269 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1188495-54f7-4986-8556-583209c26294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:05.303395 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1188495-54f7-4986-8556-583209c26294 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:05.303902 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1188495-54f7-4986-8556-583209c26294 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:05.304404 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1188495-54f7-4986-8556-583209c26294 None admin] PATH_INFO: `/v3/groups/121aa5cb19db48029d2846f8b2bf90de` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:05.309723 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1188495-54f7-4986-8556-583209c26294 None admin] RBAC: Authorizing `identity:get_group(group_id=121aa5cb19db48029d2846f8b2bf90de)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:05.312409 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1188495-54f7-4986-8556-583209c26294 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:05.315704 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 194/387] 10.222.0.55 () {56 vars in 1180 bytes} [Wed Aug 24 08:34:04 2022] GET /identity/v3/groups/121aa5cb19db48029d2846f8b2bf90de => generated 226 bytes in 406 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:34:05.324509 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8ff60b-505b-4cc9-98e0-01ff471fbb20 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:05.537989 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8ff60b-505b-4cc9-98e0-01ff471fbb20 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:05.593166 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:05.597782 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8526fa6-d0e5-4129-82fa-48af2cf2140b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:05.598441 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8526fa6-d0e5-4129-82fa-48af2cf2140b None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:05.598769 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8526fa6-d0e5-4129-82fa-48af2cf2140b None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:05.599049 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8526fa6-d0e5-4129-82fa-48af2cf2140b None admin] PATH_INFO: `/v3/projects/f8b7cfbd340b4753bd898ddc0a462a03` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:05.600430 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8526fa6-d0e5-4129-82fa-48af2cf2140b None admin] RBAC: Authorizing `identity:get_project(project_id=f8b7cfbd340b4753bd898ddc0a462a03)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:05.601970 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8526fa6-d0e5-4129-82fa-48af2cf2140b None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:05.603789 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 194/388] 10.222.0.55 () {56 vars in 1186 bytes} [Wed Aug 24 08:34:05 2022] GET /identity/v3/projects/f8b7cfbd340b4753bd898ddc0a462a03 => generated 298 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:05.611610 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1188495-54f7-4986-8556-583209c26294 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:05.757034 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1188495-54f7-4986-8556-583209c26294 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:05.793587 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:05.808602 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5a96c7-a03e-4e0b-8152-15490ec885fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:05.809294 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5a96c7-a03e-4e0b-8152-15490ec885fc None admin] REQUEST_METHOD: `PUT` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:05.809572 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5a96c7-a03e-4e0b-8152-15490ec885fc None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:05.809849 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5a96c7-a03e-4e0b-8152-15490ec885fc None admin] PATH_INFO: `/v3/projects/f8b7cfbd340b4753bd898ddc0a462a03/groups/121aa5cb19db48029d2846f8b2bf90de/roles/b674fd0a592e4e0cada811bba5ceea4d` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:05.812025 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5a96c7-a03e-4e0b-8152-15490ec885fc None admin] RBAC: Authorizing `identity:create_grant(project_id=f8b7cfbd340b4753bd898ddc0a462a03, group_id=121aa5cb19db48029d2846f8b2bf90de, role_id=b674fd0a592e4e0cada811bba5ceea4d)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:05.813565 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5a96c7-a03e-4e0b-8152-15490ec885fc None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:05.916791 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 195/389] 10.222.0.55 () {58 vars in 1442 bytes} [Wed Aug 24 08:34:05 2022] PUT /identity/v3/projects/f8b7cfbd340b4753bd898ddc0a462a03/groups/121aa5cb19db48029d2846f8b2bf90de/roles/b674fd0a592e4e0cada811bba5ceea4d => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:34:05.988395 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8526fa6-d0e5-4129-82fa-48af2cf2140b None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:06.196859 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8526fa6-d0e5-4129-82fa-48af2cf2140b None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:06.276718 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb98d7aa-5715-4229-b767-d0ada904f3c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:06.277784 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb98d7aa-5715-4229-b767-d0ada904f3c5 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:06.278237 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb98d7aa-5715-4229-b767-d0ada904f3c5 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:06.278680 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb98d7aa-5715-4229-b767-d0ada904f3c5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:06.298068 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb98d7aa-5715-4229-b767-d0ada904f3c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:34:06.308517 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-bb98d7aa-5715-4229-b767-d0ada904f3c5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 08:34:06.308517 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:06.308517 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:06.308517 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:06.308517 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:06.308517 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:06.308517 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:06.308517 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:06.308517 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:06.308517 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:06.308517 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:06.308517 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:06.308517 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:34:06.308517 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:34:06.308517 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:34:06.308517 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:34:06.309455 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:34:06.309455 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:34:06.309455 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:34:06.309455 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:34:06.309455 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:06.309455 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:06.309455 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:06.309455 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:06.309455 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:06.309455 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:06.309455 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:06.309455 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:06.309455 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:06.309455 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:06.309455 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:06.309455 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:06.310299 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:06.310299 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:06.310299 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:06.310299 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:06.310299 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:34:06.310299 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:34:06.310299 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:34:06.310299 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:34:06.310299 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:34:06.310299 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:34:06.310299 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 08:34:06.310299 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:34:06.312408 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 195/390] 10.222.0.55 () {56 vars in 1114 bytes} [Wed Aug 24 08:34:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:06.319615 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5a96c7-a03e-4e0b-8152-15490ec885fc None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:06.547544 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5a96c7-a03e-4e0b-8152-15490ec885fc None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:06.571697 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfa206a-278f-4a93-aa21-0d4b64635ff6 None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:06.572186 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfa206a-278f-4a93-aa21-0d4b64635ff6 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:06.572679 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfa206a-278f-4a93-aa21-0d4b64635ff6 None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:06.583888 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 196/391] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:34:06 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:34:06.586609 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6fe4cc-cf50-4728-abe8-39eb09620c60 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:06.587102 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6fe4cc-cf50-4728-abe8-39eb09620c60 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:06.587568 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6fe4cc-cf50-4728-abe8-39eb09620c60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:06.629009 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-8a6fe4cc-cf50-4728-abe8-39eb09620c60 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:06.630237 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-8a6fe4cc-cf50-4728-abe8-39eb09620c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:06.635060 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d08e37-e68c-49ac-8c14-77afbc93bda1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:06.636106 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d08e37-e68c-49ac-8c14-77afbc93bda1 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:06.636585 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d08e37-e68c-49ac-8c14-77afbc93bda1 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:06.637054 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d08e37-e68c-49ac-8c14-77afbc93bda1 None admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:06.639605 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d08e37-e68c-49ac-8c14-77afbc93bda1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:06.640152 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d08e37-e68c-49ac-8c14-77afbc93bda1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:06.642793 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d08e37-e68c-49ac-8c14-77afbc93bda1 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:06.666579 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 196/392] 10.222.0.55 () {56 vars in 1111 bytes} [Wed Aug 24 08:34:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:34:06.675628 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d08e37-e68c-49ac-8c14-77afbc93bda1 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:06.801258 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:06.817263 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:06.866330 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 197/393] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:34:06.917904 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b317d5-8c1c-4515-b0a8-484dd789201c None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:06.918281 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b317d5-8c1c-4515-b0a8-484dd789201c None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:06.918539 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b317d5-8c1c-4515-b0a8-484dd789201c None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:06.920191 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 198/394] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:34:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:34:06.926043 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1501c4b-fbb6-44cc-a9bc-d597ba21f1f0 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:06.926432 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1501c4b-fbb6-44cc-a9bc-d597ba21f1f0 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:06.926606 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1501c4b-fbb6-44cc-a9bc-d597ba21f1f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:06.935040 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d08e37-e68c-49ac-8c14-77afbc93bda1 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:06.971286 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-b1501c4b-fbb6-44cc-a9bc-d597ba21f1f0 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:06.972500 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-b1501c4b-fbb6-44cc-a9bc-d597ba21f1f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:07.015587 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e04ad74-e06e-46a6-82e8-56f5cf5716d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:07.016298 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e04ad74-e06e-46a6-82e8-56f5cf5716d3 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:07.016618 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e04ad74-e06e-46a6-82e8-56f5cf5716d3 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:07.016895 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e04ad74-e06e-46a6-82e8-56f5cf5716d3 None admin] PATH_INFO: `/v3/groups/121aa5cb19db48029d2846f8b2bf90de` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:07.018839 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e04ad74-e06e-46a6-82e8-56f5cf5716d3 None admin] RBAC: Authorizing `identity:get_group(group_id=121aa5cb19db48029d2846f8b2bf90de)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:07.020289 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e04ad74-e06e-46a6-82e8-56f5cf5716d3 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:07.023281 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 197/395] 10.222.0.55 () {56 vars in 1180 bytes} [Wed Aug 24 08:34:06 2022] GET /identity/v3/groups/121aa5cb19db48029d2846f8b2bf90de => generated 226 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:07.029967 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e04ad74-e06e-46a6-82e8-56f5cf5716d3 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:07.084417 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 199/396] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:34:07.086025 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1a63d0-ee7f-42bd-b229-deb7d2a5cd7d None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:07.086297 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1a63d0-ee7f-42bd-b229-deb7d2a5cd7d None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:07.086571 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1a63d0-ee7f-42bd-b229-deb7d2a5cd7d None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:07.092855 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 200/397] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:34:07 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:34:07.094696 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8870bf2-9efc-45f2-868c-9e1e2e08d93c None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:07.095013 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8870bf2-9efc-45f2-868c-9e1e2e08d93c None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:07.095280 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8870bf2-9efc-45f2-868c-9e1e2e08d93c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:07.121119 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-d8870bf2-9efc-45f2-868c-9e1e2e08d93c None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:07.122476 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-d8870bf2-9efc-45f2-868c-9e1e2e08d93c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:07.184908 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e04ad74-e06e-46a6-82e8-56f5cf5716d3 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:07.224983 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:07.242584 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613083b1-8c98-42e3-80a5-98ce697be931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:07.243288 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613083b1-8c98-42e3-80a5-98ce697be931 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:07.243572 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613083b1-8c98-42e3-80a5-98ce697be931 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:07.243846 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613083b1-8c98-42e3-80a5-98ce697be931 None admin] PATH_INFO: `/v3/projects/f8b7cfbd340b4753bd898ddc0a462a03` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:07.245390 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613083b1-8c98-42e3-80a5-98ce697be931 None admin] RBAC: Authorizing `identity:get_project(project_id=f8b7cfbd340b4753bd898ddc0a462a03)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:07.246970 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613083b1-8c98-42e3-80a5-98ce697be931 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:07.247426 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 201/398] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:34:07.248647 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8870bf2-9efc-45f2-868c-9e1e2e08d93c None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:07.256890 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 198/399] 10.222.0.55 () {56 vars in 1186 bytes} [Wed Aug 24 08:34:07 2022] GET /identity/v3/projects/f8b7cfbd340b4753bd898ddc0a462a03 => generated 298 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:34:07.264029 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613083b1-8c98-42e3-80a5-98ce697be931 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:07.344889 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:07.431162 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8870bf2-9efc-45f2-868c-9e1e2e08d93c None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:07.458391 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613083b1-8c98-42e3-80a5-98ce697be931 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:07.480419 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f33413-82c5-496c-90e8-a0f53c7497fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:07.481353 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f33413-82c5-496c-90e8-a0f53c7497fc None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:07.481653 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f33413-82c5-496c-90e8-a0f53c7497fc None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:07.481925 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f33413-82c5-496c-90e8-a0f53c7497fc None admin] PATH_INFO: `/v3/roles/member` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:07.497854 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f33413-82c5-496c-90e8-a0f53c7497fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:34:07.503339 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca90fc7-d426-4446-8d18-1a633d0dd891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:07.504033 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-98f33413-82c5-496c-90e8-a0f53c7497fc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:34:07.504033 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:07.504033 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:07.504033 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:07.504033 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:07.504033 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:07.504033 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:07.504033 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:07.504033 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:07.504033 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:07.504033 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:07.504033 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:07.504033 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:34:07.504033 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:34:07.504033 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:34:07.504033 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:34:07.505018 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:34:07.505018 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:34:07.505018 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:34:07.505018 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:34:07.505018 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:07.505018 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:07.505018 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:07.505018 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:07.505018 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:07.505018 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:07.505018 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:07.505018 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:07.505018 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:07.505018 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:07.505018 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:07.505018 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:07.506114 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:07.506114 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:07.506114 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:07.506114 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:07.506114 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:34:07.506114 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:34:07.506114 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:34:07.506114 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:34:07.506114 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:34:07.506114 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:34:07.506114 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:34:07.506114 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:34:07.506969 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca90fc7-d426-4446-8d18-1a633d0dd891 None admin] REQUEST_METHOD: `PUT` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:07.506969 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca90fc7-d426-4446-8d18-1a633d0dd891 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:07.506969 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca90fc7-d426-4446-8d18-1a633d0dd891 None admin] PATH_INFO: `/v3/projects/f8b7cfbd340b4753bd898ddc0a462a03/groups/121aa5cb19db48029d2846f8b2bf90de/roles/8b5ad96041894155b0138a676ead607e` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:07.507175 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 202/400] 10.222.0.55 () {56 vars in 1099 bytes} [Wed Aug 24 08:34:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:07.507175 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f33413-82c5-496c-90e8-a0f53c7497fc None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:07.512774 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca90fc7-d426-4446-8d18-1a633d0dd891 None admin] RBAC: Authorizing `identity:create_grant(project_id=f8b7cfbd340b4753bd898ddc0a462a03, group_id=121aa5cb19db48029d2846f8b2bf90de, role_id=8b5ad96041894155b0138a676ead607e)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:07.514475 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca90fc7-d426-4446-8d18-1a633d0dd891 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:07.589397 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 199/401] 10.222.0.55 () {58 vars in 1442 bytes} [Wed Aug 24 08:34:07 2022] PUT /identity/v3/projects/f8b7cfbd340b4753bd898ddc0a462a03/groups/121aa5cb19db48029d2846f8b2bf90de/roles/8b5ad96041894155b0138a676ead607e => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:34:07.590736 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca90fc7-d426-4446-8d18-1a633d0dd891 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:07.636656 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f33413-82c5-496c-90e8-a0f53c7497fc None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:07.648020 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:07.655841 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:07.677631 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeeb9dc-5330-48fa-9ea2-d61c557c6faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:07.678449 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeeb9dc-5330-48fa-9ea2-d61c557c6faa None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:07.678830 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeeb9dc-5330-48fa-9ea2-d61c557c6faa None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:07.679206 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeeb9dc-5330-48fa-9ea2-d61c557c6faa None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:07.692889 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aeeb9dc-5330-48fa-9ea2-d61c557c6faa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:34:07.698999 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-9aeeb9dc-5330-48fa-9ea2-d61c557c6faa None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 08:34:07.698999 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:07.698999 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:07.698999 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:07.698999 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:07.698999 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:07.698999 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:07.698999 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:07.698999 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:07.698999 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:07.698999 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:07.698999 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:07.698999 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:34:07.698999 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:34:07.698999 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:34:07.698999 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:34:07.700055 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:34:07.700055 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:34:07.700055 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:34:07.700055 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:34:07.700055 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:07.700055 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:07.700055 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:07.700055 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:07.700055 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:07.700055 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:07.700055 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:07.700055 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:07.700055 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:07.700055 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:07.700055 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:07.700055 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:07.700936 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:07.700936 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:07.700936 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:07.700936 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:07.700936 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:34:07.700936 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:34:07.700936 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:34:07.700936 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:34:07.700936 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:34:07.700936 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:34:07.700936 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 08:34:07.700936 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:34:07.700936 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 203/402] 10.222.0.55 () {56 vars in 1114 bytes} [Wed Aug 24 08:34:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:07.701878 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeeb9dc-5330-48fa-9ea2-d61c557c6faa None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:07.739511 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca90fc7-d426-4446-8d18-1a633d0dd891 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:07.783871 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06421f9-75a3-4713-84d6-9d5701d1178d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:07.783871 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06421f9-75a3-4713-84d6-9d5701d1178d None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:07.783871 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06421f9-75a3-4713-84d6-9d5701d1178d None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:07.784232 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06421f9-75a3-4713-84d6-9d5701d1178d None admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:07.785031 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06421f9-75a3-4713-84d6-9d5701d1178d None admin] RBAC: Adding query filter params (name=member) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:07.785345 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06421f9-75a3-4713-84d6-9d5701d1178d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:07.786910 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06421f9-75a3-4713-84d6-9d5701d1178d None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:07.800417 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 200/403] 10.222.0.55 () {56 vars in 1101 bytes} [Wed Aug 24 08:34:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:07.801376 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06421f9-75a3-4713-84d6-9d5701d1178d None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:07.830180 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeeb9dc-5330-48fa-9ea2-d61c557c6faa None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:07.873075 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67407e88-a1a7-4a35-9c0a-3feb4552c968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:07.874521 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67407e88-a1a7-4a35-9c0a-3feb4552c968 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:07.875017 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67407e88-a1a7-4a35-9c0a-3feb4552c968 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:07.875739 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67407e88-a1a7-4a35-9c0a-3feb4552c968 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:07.888449 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67407e88-a1a7-4a35-9c0a-3feb4552c968 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:34:07.894893 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-67407e88-a1a7-4a35-9c0a-3feb4552c968 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:34:07.894893 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:07.894893 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:07.894893 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:07.894893 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:07.894893 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:07.894893 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:07.894893 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:07.894893 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:07.894893 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:07.894893 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:07.894893 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:07.894893 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:34:07.894893 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:34:07.894893 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:34:07.894893 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:34:07.895844 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:34:07.895844 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:34:07.895844 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:34:07.895844 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:34:07.895844 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:07.895844 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:07.895844 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:07.895844 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:07.895844 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:07.895844 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:07.895844 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:07.895844 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:07.895844 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:07.895844 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:07.895844 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:07.895844 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:07.896703 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:07.896703 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:07.896703 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:07.896703 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:07.896703 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:34:07.896703 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:34:07.896703 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:34:07.896703 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:34:07.896703 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:34:07.896703 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:34:07.896703 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:34:07.896703 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:34:07.899506 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 204/404] 10.222.0.55 () {56 vars in 1096 bytes} [Wed Aug 24 08:34:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:07.900679 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67407e88-a1a7-4a35-9c0a-3feb4552c968 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:07.935320 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:07.956517 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06421f9-75a3-4713-84d6-9d5701d1178d None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:07.980838 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:08.028634 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37221517-0bec-4e1c-9249-49c91c417210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:08.029511 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67407e88-a1a7-4a35-9c0a-3feb4552c968 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:08.029705 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37221517-0bec-4e1c-9249-49c91c417210 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:08.030167 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37221517-0bec-4e1c-9249-49c91c417210 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:08.030663 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37221517-0bec-4e1c-9249-49c91c417210 None admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:08.031891 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37221517-0bec-4e1c-9249-49c91c417210 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:08.032444 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37221517-0bec-4e1c-9249-49c91c417210 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:08.035186 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37221517-0bec-4e1c-9249-49c91c417210 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:08.059428 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 201/405] 10.222.0.55 () {56 vars in 1111 bytes} [Wed Aug 24 08:34:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:08.061009 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37221517-0bec-4e1c-9249-49c91c417210 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:08.073672 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836f09fd-2f22-49da-a25e-a2b4fe884b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:08.074694 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836f09fd-2f22-49da-a25e-a2b4fe884b54 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:08.075181 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836f09fd-2f22-49da-a25e-a2b4fe884b54 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:08.075676 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836f09fd-2f22-49da-a25e-a2b4fe884b54 None admin] PATH_INFO: `/v3/groups/121aa5cb19db48029d2846f8b2bf90de` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:08.077309 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836f09fd-2f22-49da-a25e-a2b4fe884b54 None admin] RBAC: Authorizing `identity:get_group(group_id=121aa5cb19db48029d2846f8b2bf90de)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:08.079234 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836f09fd-2f22-49da-a25e-a2b4fe884b54 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:08.081269 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 205/406] 10.222.0.55 () {56 vars in 1180 bytes} [Wed Aug 24 08:34:07 2022] GET /identity/v3/groups/121aa5cb19db48029d2846f8b2bf90de => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:08.082448 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836f09fd-2f22-49da-a25e-a2b4fe884b54 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:08.195534 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37221517-0bec-4e1c-9249-49c91c417210 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:08.212271 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:08.221478 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836f09fd-2f22-49da-a25e-a2b4fe884b54 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:08.236889 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e7ff6f-0161-41d6-b99f-23b8a9c7e726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:08.237509 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e7ff6f-0161-41d6-b99f-23b8a9c7e726 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:08.237878 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e7ff6f-0161-41d6-b99f-23b8a9c7e726 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:08.238160 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e7ff6f-0161-41d6-b99f-23b8a9c7e726 None admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:08.238974 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e7ff6f-0161-41d6-b99f-23b8a9c7e726 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:08.239282 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e7ff6f-0161-41d6-b99f-23b8a9c7e726 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:08.240826 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e7ff6f-0161-41d6-b99f-23b8a9c7e726 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:08.251667 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 202/407] 10.222.0.55 () {56 vars in 1099 bytes} [Wed Aug 24 08:34:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:08.252702 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e7ff6f-0161-41d6-b99f-23b8a9c7e726 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:08.274439 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a3ecf2-3289-4c18-98d0-b54268701aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:08.275527 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a3ecf2-3289-4c18-98d0-b54268701aa0 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:08.276030 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a3ecf2-3289-4c18-98d0-b54268701aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:08.277043 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a3ecf2-3289-4c18-98d0-b54268701aa0 None admin] PATH_INFO: `/v3/groups/121aa5cb19db48029d2846f8b2bf90de` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:08.278649 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a3ecf2-3289-4c18-98d0-b54268701aa0 None admin] RBAC: Authorizing `identity:get_group(group_id=121aa5cb19db48029d2846f8b2bf90de)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:08.280405 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a3ecf2-3289-4c18-98d0-b54268701aa0 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:08.282899 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 206/408] 10.222.0.55 () {56 vars in 1180 bytes} [Wed Aug 24 08:34:08 2022] GET /identity/v3/groups/121aa5cb19db48029d2846f8b2bf90de => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:08.284023 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a3ecf2-3289-4c18-98d0-b54268701aa0 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:08.291431 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:08.375882 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e7ff6f-0161-41d6-b99f-23b8a9c7e726 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:08.405769 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a3ecf2-3289-4c18-98d0-b54268701aa0 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:08.427698 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d28a81-feee-4c1e-a109-205720f81ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:08.429243 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d28a81-feee-4c1e-a109-205720f81ba6 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:08.429780 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d28a81-feee-4c1e-a109-205720f81ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:08.430261 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d28a81-feee-4c1e-a109-205720f81ba6 None admin] PATH_INFO: `/v3/projects/61247a1049b846b79052f42b0d525259` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:08.432753 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d28a81-feee-4c1e-a109-205720f81ba6 None admin] RBAC: Authorizing `identity:get_project(project_id=61247a1049b846b79052f42b0d525259)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:08.434863 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d28a81-feee-4c1e-a109-205720f81ba6 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:08.438267 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 203/409] 10.222.0.55 () {56 vars in 1186 bytes} [Wed Aug 24 08:34:08 2022] GET /identity/v3/projects/61247a1049b846b79052f42b0d525259 => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:08.440107 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d28a81-feee-4c1e-a109-205720f81ba6 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:08.449486 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855ae79d-004c-430a-b038-36086bdcc264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:08.450147 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855ae79d-004c-430a-b038-36086bdcc264 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:08.450432 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855ae79d-004c-430a-b038-36086bdcc264 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:08.450710 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855ae79d-004c-430a-b038-36086bdcc264 None admin] PATH_INFO: `/v3/groups/a37059af981c4b8b8bee1a21f0f63ebb` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:08.451983 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855ae79d-004c-430a-b038-36086bdcc264 None admin] RBAC: Authorizing `identity:get_group(group_id=a37059af981c4b8b8bee1a21f0f63ebb)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:08.453799 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855ae79d-004c-430a-b038-36086bdcc264 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:08.455508 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 207/410] 10.222.0.55 () {56 vars in 1180 bytes} [Wed Aug 24 08:34:08 2022] GET /identity/v3/groups/a37059af981c4b8b8bee1a21f0f63ebb => generated 229 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:08.456766 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855ae79d-004c-430a-b038-36086bdcc264 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:08.495609 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:08.589127 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855ae79d-004c-430a-b038-36086bdcc264 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:08.592484 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d28a81-feee-4c1e-a109-205720f81ba6 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:08.594362 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:08.630924 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1cb21a-71a6-4ef1-9640-3751b3ff6e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:08.631804 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1cb21a-71a6-4ef1-9640-3751b3ff6e1c None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:08.632159 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1cb21a-71a6-4ef1-9640-3751b3ff6e1c None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:08.632873 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1cb21a-71a6-4ef1-9640-3751b3ff6e1c None admin] PATH_INFO: `/v3/projects/61247a1049b846b79052f42b0d525259` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:08.634088 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1cb21a-71a6-4ef1-9640-3751b3ff6e1c None admin] RBAC: Authorizing `identity:get_project(project_id=61247a1049b846b79052f42b0d525259)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:08.635947 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1cb21a-71a6-4ef1-9640-3751b3ff6e1c None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:08.638550 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 204/411] 10.222.0.55 () {56 vars in 1186 bytes} [Wed Aug 24 08:34:08 2022] GET /identity/v3/projects/61247a1049b846b79052f42b0d525259 => generated 302 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:08.639454 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b570d27c-d7bc-478a-90d3-c5ddd6803762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:08.639940 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1cb21a-71a6-4ef1-9640-3751b3ff6e1c None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:08.640951 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b570d27c-d7bc-478a-90d3-c5ddd6803762 None admin] REQUEST_METHOD: `PUT` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:08.641382 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b570d27c-d7bc-478a-90d3-c5ddd6803762 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:08.641816 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b570d27c-d7bc-478a-90d3-c5ddd6803762 None admin] PATH_INFO: `/v3/projects/61247a1049b846b79052f42b0d525259/groups/121aa5cb19db48029d2846f8b2bf90de/roles/b674fd0a592e4e0cada811bba5ceea4d` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:08.645116 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b570d27c-d7bc-478a-90d3-c5ddd6803762 None admin] RBAC: Authorizing `identity:create_grant(project_id=61247a1049b846b79052f42b0d525259, group_id=121aa5cb19db48029d2846f8b2bf90de, role_id=b674fd0a592e4e0cada811bba5ceea4d)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:08.646606 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b570d27c-d7bc-478a-90d3-c5ddd6803762 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:08.728729 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 208/412] 10.222.0.55 () {58 vars in 1442 bytes} [Wed Aug 24 08:34:08 2022] PUT /identity/v3/projects/61247a1049b846b79052f42b0d525259/groups/121aa5cb19db48029d2846f8b2bf90de/roles/b674fd0a592e4e0cada811bba5ceea4d => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:34:08.730017 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b570d27c-d7bc-478a-90d3-c5ddd6803762 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:08.795979 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1cb21a-71a6-4ef1-9640-3751b3ff6e1c None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:08.868598 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86373ef-a5a3-4370-9a72-e2224173654c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:08.869305 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86373ef-a5a3-4370-9a72-e2224173654c None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:08.869592 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86373ef-a5a3-4370-9a72-e2224173654c None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:08.869873 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86373ef-a5a3-4370-9a72-e2224173654c None admin] PATH_INFO: `/v3/projects/6ad87fa1813247aba83e93f886f34eb3` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:08.871336 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:08.871625 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86373ef-a5a3-4370-9a72-e2224173654c None admin] RBAC: Authorizing `identity:get_project(project_id=6ad87fa1813247aba83e93f886f34eb3)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:08.873987 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86373ef-a5a3-4370-9a72-e2224173654c None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:08.876605 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 205/413] 10.222.0.55 () {56 vars in 1186 bytes} [Wed Aug 24 08:34:08 2022] GET /identity/v3/projects/6ad87fa1813247aba83e93f886f34eb3 => generated 344 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:08.885462 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86373ef-a5a3-4370-9a72-e2224173654c None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:08.893029 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b570d27c-d7bc-478a-90d3-c5ddd6803762 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:08.935717 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ece7a07-48f4-4dee-a8d4-7554568f4569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:08.936583 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ece7a07-48f4-4dee-a8d4-7554568f4569 None admin] REQUEST_METHOD: `PUT` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:08.936870 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ece7a07-48f4-4dee-a8d4-7554568f4569 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:08.937143 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ece7a07-48f4-4dee-a8d4-7554568f4569 None admin] PATH_INFO: `/v3/projects/61247a1049b846b79052f42b0d525259/groups/121aa5cb19db48029d2846f8b2bf90de/roles/8b5ad96041894155b0138a676ead607e` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:08.940673 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ece7a07-48f4-4dee-a8d4-7554568f4569 None admin] RBAC: Authorizing `identity:create_grant(project_id=61247a1049b846b79052f42b0d525259, group_id=121aa5cb19db48029d2846f8b2bf90de, role_id=8b5ad96041894155b0138a676ead607e)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:08.941779 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:08.942907 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ece7a07-48f4-4dee-a8d4-7554568f4569 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:09.044917 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 209/414] 10.222.0.55 () {58 vars in 1442 bytes} [Wed Aug 24 08:34:08 2022] PUT /identity/v3/projects/61247a1049b846b79052f42b0d525259/groups/121aa5cb19db48029d2846f8b2bf90de/roles/8b5ad96041894155b0138a676ead607e => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:34:09.048890 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8a6185-b0c6-4b41-932a-a7cbbbfb199b None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:09.049419 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8a6185-b0c6-4b41-932a-a7cbbbfb199b None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:09.049900 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8a6185-b0c6-4b41-932a-a7cbbbfb199b None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:09.053267 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 210/415] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:34:09 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:34:09.056690 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86373ef-a5a3-4370-9a72-e2224173654c None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:09.061303 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eeaa35-7795-45cc-b679-a817b548557c None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:09.061774 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eeaa35-7795-45cc-b679-a817b548557c None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:09.062244 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eeaa35-7795-45cc-b679-a817b548557c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:09.119175 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-65eeaa35-7795-45cc-b679-a817b548557c None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:09.120490 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-65eeaa35-7795-45cc-b679-a817b548557c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:09.161553 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e343d28-9793-482a-8390-5ccf99339734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:09.162879 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e343d28-9793-482a-8390-5ccf99339734 None admin] REQUEST_METHOD: `PUT` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:09.163377 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e343d28-9793-482a-8390-5ccf99339734 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:09.163846 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e343d28-9793-482a-8390-5ccf99339734 None admin] PATH_INFO: `/v3/projects/6ad87fa1813247aba83e93f886f34eb3/groups/a37059af981c4b8b8bee1a21f0f63ebb/roles/b870cd1acaa744e1ae1d58b9ac283821` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:09.171188 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e343d28-9793-482a-8390-5ccf99339734 None admin] RBAC: Authorizing `identity:create_grant(project_id=6ad87fa1813247aba83e93f886f34eb3, group_id=a37059af981c4b8b8bee1a21f0f63ebb, role_id=b870cd1acaa744e1ae1d58b9ac283821)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:09.178574 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e343d28-9793-482a-8390-5ccf99339734 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:09.272750 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 206/416] 10.222.0.55 () {58 vars in 1442 bytes} [Wed Aug 24 08:34:08 2022] PUT /identity/v3/projects/6ad87fa1813247aba83e93f886f34eb3/groups/a37059af981c4b8b8bee1a21f0f63ebb/roles/b870cd1acaa744e1ae1d58b9ac283821 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:34:09.282984 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 211/417] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:34:09.885010 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e343d28-9793-482a-8390-5ccf99339734 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:10.097402 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e343d28-9793-482a-8390-5ccf99339734 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:10.104346 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:10.206321 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3dd0de-4fc2-46fa-8125-3bf849e6855e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:10.206321 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3dd0de-4fc2-46fa-8125-3bf849e6855e None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:10.206321 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3dd0de-4fc2-46fa-8125-3bf849e6855e None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:10.206321 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3dd0de-4fc2-46fa-8125-3bf849e6855e None admin] PATH_INFO: `/v3/roles/member` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:10.233354 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43047e3-4c72-4d4f-9ad3-ebcc9c373b9b None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:10.234320 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43047e3-4c72-4d4f-9ad3-ebcc9c373b9b None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:10.234810 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43047e3-4c72-4d4f-9ad3-ebcc9c373b9b None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:10.236768 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 212/418] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:34:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:34:10.241369 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3dd0de-4fc2-46fa-8125-3bf849e6855e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:34:10.243164 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd21b72-f5ac-4aee-a927-4e75aff0c702 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:10.243664 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd21b72-f5ac-4aee-a927-4e75aff0c702 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:10.244152 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd21b72-f5ac-4aee-a927-4e75aff0c702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:10.251764 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-3b3dd0de-4fc2-46fa-8125-3bf849e6855e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:34:10.251764 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:10.251764 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:10.251764 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:10.251764 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:10.251764 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:10.251764 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:10.251764 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:10.251764 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:10.251764 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:10.251764 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:10.251764 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:10.251764 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:34:10.251764 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:34:10.251764 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:34:10.251764 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:34:10.252958 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:34:10.252958 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:34:10.252958 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:34:10.252958 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:34:10.252958 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:10.252958 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:10.252958 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:10.252958 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:10.252958 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:10.252958 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:10.252958 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:10.252958 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:10.252958 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:10.252958 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:10.252958 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:10.252958 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:10.253753 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:10.253753 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:10.253753 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:10.253753 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:10.253753 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:34:10.253753 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:34:10.253753 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:34:10.253753 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:34:10.253753 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:34:10.253753 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:34:10.253753 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:34:10.253753 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:34:10.256637 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 207/419] 10.222.0.55 () {56 vars in 1099 bytes} [Wed Aug 24 08:34:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 370 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:10.264799 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3dd0de-4fc2-46fa-8125-3bf849e6855e None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:10.277946 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-1fd21b72-f5ac-4aee-a927-4e75aff0c702 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:10.282982 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-1fd21b72-f5ac-4aee-a927-4e75aff0c702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:10.448671 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 213/420] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:34:10.470436 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3dd0de-4fc2-46fa-8125-3bf849e6855e None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:10.510874 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ab387c-6c05-45a3-96f2-ce2d73d67cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:10.511922 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ab387c-6c05-45a3-96f2-ce2d73d67cd9 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:10.512403 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ab387c-6c05-45a3-96f2-ce2d73d67cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:10.512906 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ab387c-6c05-45a3-96f2-ce2d73d67cd9 None admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:10.513804 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ab387c-6c05-45a3-96f2-ce2d73d67cd9 None admin] RBAC: Adding query filter params (name=member) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:10.514320 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ab387c-6c05-45a3-96f2-ce2d73d67cd9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:10.516062 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ab387c-6c05-45a3-96f2-ce2d73d67cd9 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:10.527464 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 208/421] 10.222.0.55 () {56 vars in 1101 bytes} [Wed Aug 24 08:34:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:34:10.535299 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd21b72-f5ac-4aee-a927-4e75aff0c702 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:10.561472 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:10.674499 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd21b72-f5ac-4aee-a927-4e75aff0c702 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:10.715912 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dedf51-9f95-42d5-bd35-03f553f22e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:10.716978 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dedf51-9f95-42d5-bd35-03f553f22e04 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:10.717519 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dedf51-9f95-42d5-bd35-03f553f22e04 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:10.718005 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dedf51-9f95-42d5-bd35-03f553f22e04 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:10.808951 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dedf51-9f95-42d5-bd35-03f553f22e04 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:10.810879 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dedf51-9f95-42d5-bd35-03f553f22e04 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:10.828217 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:10.996757 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 214/422] 10.222.0.55 () {58 vars in 1282 bytes} [Wed Aug 24 08:34:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 08:34:11.005072 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ab387c-6c05-45a3-96f2-ce2d73d67cd9 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:11.067060 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:11.216407 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dedf51-9f95-42d5-bd35-03f553f22e04 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:11.302637 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ab387c-6c05-45a3-96f2-ce2d73d67cd9 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:11.426085 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed1d19a-50dd-4f64-be89-8ab736cdc6f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:11.427757 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed1d19a-50dd-4f64-be89-8ab736cdc6f4 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:11.428591 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed1d19a-50dd-4f64-be89-8ab736cdc6f4 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:11.429071 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed1d19a-50dd-4f64-be89-8ab736cdc6f4 None admin] PATH_INFO: `/v3/projects/f8b7cfbd340b4753bd898ddc0a462a03` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:11.431091 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed1d19a-50dd-4f64-be89-8ab736cdc6f4 None admin] RBAC: Authorizing `identity:get_project(project_id=f8b7cfbd340b4753bd898ddc0a462a03)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:11.434449 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed1d19a-50dd-4f64-be89-8ab736cdc6f4 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:11.437976 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 209/423] 10.222.0.55 () {56 vars in 1186 bytes} [Wed Aug 24 08:34:11 2022] GET /identity/v3/projects/f8b7cfbd340b4753bd898ddc0a462a03 => generated 298 bytes in 433 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:34:11.448898 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed1d19a-50dd-4f64-be89-8ab736cdc6f4 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:11.521195 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dedf51-9f95-42d5-bd35-03f553f22e04 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:11.561096 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:11.621289 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e2ac79-370e-42cc-a50a-fdb8a81a350d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:11.622447 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e2ac79-370e-42cc-a50a-fdb8a81a350d None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:11.623020 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e2ac79-370e-42cc-a50a-fdb8a81a350d None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:11.623513 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e2ac79-370e-42cc-a50a-fdb8a81a350d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:11.653360 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:11.654490 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e2ac79-370e-42cc-a50a-fdb8a81a350d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:34:11.659984 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-e8e2ac79-370e-42cc-a50a-fdb8a81a350d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 08:34:11.659984 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:11.659984 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:11.659984 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:11.659984 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:11.659984 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:11.659984 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:11.659984 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:11.659984 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:11.659984 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:11.659984 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:11.659984 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:11.659984 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:34:11.659984 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:34:11.659984 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:34:11.659984 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:34:11.664182 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:34:11.664182 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:34:11.664182 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:34:11.664182 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:34:11.664182 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:11.664182 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:11.664182 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:11.664182 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:11.664182 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:11.664182 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:11.664182 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:11.664182 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:11.664182 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:11.664182 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:11.664182 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:11.664182 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:11.665233 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:11.665233 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:11.665233 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:11.665233 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:11.665233 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:34:11.665233 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:34:11.665233 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:34:11.665233 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:34:11.665233 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:34:11.665233 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:34:11.665233 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 08:34:11.665233 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:34:11.665233 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 215/424] 10.222.0.55 () {56 vars in 1114 bytes} [Wed Aug 24 08:34:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 446 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:11.671578 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e2ac79-370e-42cc-a50a-fdb8a81a350d None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:11.685952 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed1d19a-50dd-4f64-be89-8ab736cdc6f4 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:11.737282 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49e2726-733b-456a-89ee-ec3108d1669e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:11.738842 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49e2726-733b-456a-89ee-ec3108d1669e None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:11.739601 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49e2726-733b-456a-89ee-ec3108d1669e None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:11.740252 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49e2726-733b-456a-89ee-ec3108d1669e None admin] PATH_INFO: `/v3/groups/121aa5cb19db48029d2846f8b2bf90de` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:11.743644 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49e2726-733b-456a-89ee-ec3108d1669e None admin] RBAC: Authorizing `identity:get_group(group_id=121aa5cb19db48029d2846f8b2bf90de)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:11.746520 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49e2726-733b-456a-89ee-ec3108d1669e None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:11.749944 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 210/425] 10.222.0.55 () {56 vars in 1180 bytes} [Wed Aug 24 08:34:11 2022] GET /identity/v3/groups/121aa5cb19db48029d2846f8b2bf90de => generated 226 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:34:11.760059 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49e2726-733b-456a-89ee-ec3108d1669e None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:11.878821 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49e2726-733b-456a-89ee-ec3108d1669e None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:11.927606 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c760f7f-4137-4fb5-ae08-741db7dca3c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:11.928677 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c760f7f-4137-4fb5-ae08-741db7dca3c2 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:11.929197 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e2ac79-370e-42cc-a50a-fdb8a81a350d None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:11.929327 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c760f7f-4137-4fb5-ae08-741db7dca3c2 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:11.929804 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c760f7f-4137-4fb5-ae08-741db7dca3c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:11.930878 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c760f7f-4137-4fb5-ae08-741db7dca3c2 None admin] RBAC: Adding query filter params (group.id=121aa5cb19db48029d2846f8b2bf90de, role.id=b674fd0a592e4e0cada811bba5ceea4d, scope.project.id=f8b7cfbd340b4753bd898ddc0a462a03) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:11.931419 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c760f7f-4137-4fb5-ae08-741db7dca3c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:11.933881 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c760f7f-4137-4fb5-ae08-741db7dca3c2 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:11.955563 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 211/426] 10.222.0.55 () {56 vars in 1376 bytes} [Wed Aug 24 08:34:11 2022] GET /identity/v3/role_assignments?group.id=121aa5cb19db48029d2846f8b2bf90de&scope.project.id=f8b7cfbd340b4753bd898ddc0a462a03&role.id=b674fd0a592e4e0cada811bba5ceea4d => generated 611 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:34:12.040141 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce92dae1-ec33-4485-bd5f-38ddc3d0cc25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:12.041105 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce92dae1-ec33-4485-bd5f-38ddc3d0cc25 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:12.041529 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce92dae1-ec33-4485-bd5f-38ddc3d0cc25 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:12.041948 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce92dae1-ec33-4485-bd5f-38ddc3d0cc25 None admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:12.043111 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce92dae1-ec33-4485-bd5f-38ddc3d0cc25 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:12.043586 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce92dae1-ec33-4485-bd5f-38ddc3d0cc25 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:12.045964 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce92dae1-ec33-4485-bd5f-38ddc3d0cc25 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:12.070076 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 216/427] 10.222.0.55 () {56 vars in 1111 bytes} [Wed Aug 24 08:34:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 399 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:12.080425 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c760f7f-4137-4fb5-ae08-741db7dca3c2 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:12.104165 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a6c47f-5bf5-4abd-be37-ad65c09b7f58 None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:12.104944 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a6c47f-5bf5-4abd-be37-ad65c09b7f58 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:12.105448 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a6c47f-5bf5-4abd-be37-ad65c09b7f58 None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:12.107528 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 217/428] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:34:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:34:12.114154 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51db363-7f12-43bd-aaa7-02482ad3f3a0 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:12.114772 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51db363-7f12-43bd-aaa7-02482ad3f3a0 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:12.115280 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51db363-7f12-43bd-aaa7-02482ad3f3a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:12.138963 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-d51db363-7f12-43bd-aaa7-02482ad3f3a0 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:12.140655 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-d51db363-7f12-43bd-aaa7-02482ad3f3a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:12.160300 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:12.214001 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:12.310879 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 218/429] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:34:12.335029 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c760f7f-4137-4fb5-ae08-741db7dca3c2 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:12.383263 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafd2b84-471b-43d9-8c41-e920e2e0c401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:12.384246 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafd2b84-471b-43d9-8c41-e920e2e0c401 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:12.384828 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafd2b84-471b-43d9-8c41-e920e2e0c401 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:12.385330 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafd2b84-471b-43d9-8c41-e920e2e0c401 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:12.486321 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafd2b84-471b-43d9-8c41-e920e2e0c401 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:12.488208 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafd2b84-471b-43d9-8c41-e920e2e0c401 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:12.519905 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:12.623117 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 212/430] 10.222.0.55 () {58 vars in 1282 bytes} [Wed Aug 24 08:34:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 544 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 08:34:12.632893 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51db363-7f12-43bd-aaa7-02482ad3f3a0 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:12.786905 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafd2b84-471b-43d9-8c41-e920e2e0c401 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:12.848847 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51db363-7f12-43bd-aaa7-02482ad3f3a0 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:12.918336 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7106fe-bb6a-460a-9f33-41a6edc48d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:12.919945 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7106fe-bb6a-460a-9f33-41a6edc48d92 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:12.919945 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7106fe-bb6a-460a-9f33-41a6edc48d92 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:12.919945 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7106fe-bb6a-460a-9f33-41a6edc48d92 None admin] PATH_INFO: `/v3/projects/f8b7cfbd340b4753bd898ddc0a462a03` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:12.920884 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7106fe-bb6a-460a-9f33-41a6edc48d92 None admin] RBAC: Authorizing `identity:get_project(project_id=f8b7cfbd340b4753bd898ddc0a462a03)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:12.922343 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7106fe-bb6a-460a-9f33-41a6edc48d92 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:12.924025 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 219/431] 10.222.0.55 () {56 vars in 1186 bytes} [Wed Aug 24 08:34:12 2022] GET /identity/v3/projects/f8b7cfbd340b4753bd898ddc0a462a03 => generated 298 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:12.925864 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff8f28c-6c23-4a4c-bff2-c45f96831d0d None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:12.925864 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff8f28c-6c23-4a4c-bff2-c45f96831d0d None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:12.925864 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff8f28c-6c23-4a4c-bff2-c45f96831d0d None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:12.927342 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 220/432] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:34:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:34:12.933163 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44e623e-5a34-4165-94c0-c2b20b3f5b47 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:12.933418 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44e623e-5a34-4165-94c0-c2b20b3f5b47 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:12.933948 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44e623e-5a34-4165-94c0-c2b20b3f5b47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:12.934421 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafd2b84-471b-43d9-8c41-e920e2e0c401 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:12.964946 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:12.969740 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-d44e623e-5a34-4165-94c0-c2b20b3f5b47 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:12.972933 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-d44e623e-5a34-4165-94c0-c2b20b3f5b47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:12.981716 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d4c7a6-86c4-42fb-80da-3bb653506de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:12.982367 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d4c7a6-86c4-42fb-80da-3bb653506de6 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:12.989527 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d4c7a6-86c4-42fb-80da-3bb653506de6 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:12.989527 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d4c7a6-86c4-42fb-80da-3bb653506de6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:12.997777 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d4c7a6-86c4-42fb-80da-3bb653506de6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:34:13.000312 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-d9d4c7a6-86c4-42fb-80da-3bb653506de6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:34:13.000312 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:13.000312 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:13.000312 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:13.000312 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:13.000312 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:13.000312 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:13.000312 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:13.000312 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:13.000312 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:13.000312 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:13.000312 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:13.000312 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:34:13.000312 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:34:13.000312 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:34:13.000312 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:34:13.002648 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:34:13.002648 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:34:13.002648 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:34:13.002648 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:34:13.002648 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:13.002648 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:13.002648 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:13.002648 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:13.002648 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:13.002648 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:13.002648 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:13.002648 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:13.002648 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:13.002648 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:13.002648 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:13.002648 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:13.004961 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:13.004961 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:13.004961 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:13.004961 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:13.004961 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:34:13.004961 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:34:13.004961 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:34:13.004961 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:34:13.004961 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:34:13.004961 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:34:13.004961 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:34:13.004961 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:34:13.007239 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 213/433] 10.222.0.55 () {56 vars in 1099 bytes} [Wed Aug 24 08:34:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:13.007772 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d4c7a6-86c4-42fb-80da-3bb653506de6 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:13.039917 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:13.123514 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 221/434] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 08:34:13.125080 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44e623e-5a34-4165-94c0-c2b20b3f5b47 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:13.186905 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d4c7a6-86c4-42fb-80da-3bb653506de6 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:13.262691 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4778f2-d610-48fc-9b4b-309868c5e3f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:13.263719 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4778f2-d610-48fc-9b4b-309868c5e3f1 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:13.264164 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4778f2-d610-48fc-9b4b-309868c5e3f1 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:13.264753 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4778f2-d610-48fc-9b4b-309868c5e3f1 None admin] PATH_INFO: `/v3/groups/121aa5cb19db48029d2846f8b2bf90de` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:13.266530 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4778f2-d610-48fc-9b4b-309868c5e3f1 None admin] RBAC: Authorizing `identity:get_group(group_id=121aa5cb19db48029d2846f8b2bf90de)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:13.268965 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4778f2-d610-48fc-9b4b-309868c5e3f1 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:13.280125 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 214/435] 10.222.0.55 () {56 vars in 1180 bytes} [Wed Aug 24 08:34:13 2022] GET /identity/v3/groups/121aa5cb19db48029d2846f8b2bf90de => generated 226 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:13.280125 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b123b3d-96a0-46ea-867f-e5efa2461b2c None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:13.280125 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b123b3d-96a0-46ea-867f-e5efa2461b2c None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:13.280125 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b123b3d-96a0-46ea-867f-e5efa2461b2c None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:13.280125 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 215/436] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:34:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:34:13.281978 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17a94d3-eabc-435f-a2e4-9af47bd83528 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:13.282325 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17a94d3-eabc-435f-a2e4-9af47bd83528 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:13.283255 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17a94d3-eabc-435f-a2e4-9af47bd83528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:13.303968 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44e623e-5a34-4165-94c0-c2b20b3f5b47 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:13.310710 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-e17a94d3-eabc-435f-a2e4-9af47bd83528 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:13.312628 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-e17a94d3-eabc-435f-a2e4-9af47bd83528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:13.361131 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2341965-5499-403c-9b3a-34d37f3fe4cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:13.362108 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2341965-5499-403c-9b3a-34d37f3fe4cf None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:13.362569 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2341965-5499-403c-9b3a-34d37f3fe4cf None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:13.363034 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2341965-5499-403c-9b3a-34d37f3fe4cf None admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:13.364167 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2341965-5499-403c-9b3a-34d37f3fe4cf None admin] RBAC: Adding query filter params (name=member) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:13.364706 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2341965-5499-403c-9b3a-34d37f3fe4cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:13.367072 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2341965-5499-403c-9b3a-34d37f3fe4cf None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:13.381643 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 222/437] 10.222.0.55 () {56 vars in 1101 bytes} [Wed Aug 24 08:34:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:13.383007 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2341965-5499-403c-9b3a-34d37f3fe4cf None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:13.392995 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:13.417288 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:13.423325 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 216/438] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 08:34:13.424616 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17a94d3-eabc-435f-a2e4-9af47bd83528 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:13.531595 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2341965-5499-403c-9b3a-34d37f3fe4cf None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:13.558355 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17a94d3-eabc-435f-a2e4-9af47bd83528 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:13.587802 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f321337e-e7f8-4194-b46b-fc521e74b6ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:13.588782 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f321337e-e7f8-4194-b46b-fc521e74b6ec None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:13.589281 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f321337e-e7f8-4194-b46b-fc521e74b6ec None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:13.589721 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f321337e-e7f8-4194-b46b-fc521e74b6ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:13.591004 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f321337e-e7f8-4194-b46b-fc521e74b6ec None admin] RBAC: Adding query filter params (group.id=121aa5cb19db48029d2846f8b2bf90de, role.id=8b5ad96041894155b0138a676ead607e, scope.project.id=f8b7cfbd340b4753bd898ddc0a462a03) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:13.591510 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f321337e-e7f8-4194-b46b-fc521e74b6ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:13.593833 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f321337e-e7f8-4194-b46b-fc521e74b6ec None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:13.603782 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 223/439] 10.222.0.55 () {56 vars in 1376 bytes} [Wed Aug 24 08:34:13 2022] GET /identity/v3/role_assignments?group.id=121aa5cb19db48029d2846f8b2bf90de&scope.project.id=f8b7cfbd340b4753bd898ddc0a462a03&role.id=8b5ad96041894155b0138a676ead607e => generated 611 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:13.605355 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f321337e-e7f8-4194-b46b-fc521e74b6ec None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:13.618760 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7271ba99-15e8-4dda-aa24-3bbd918a8813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:13.619812 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7271ba99-15e8-4dda-aa24-3bbd918a8813 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:13.620894 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7271ba99-15e8-4dda-aa24-3bbd918a8813 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:13.622126 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7271ba99-15e8-4dda-aa24-3bbd918a8813 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:13.695432 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:13.733887 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7271ba99-15e8-4dda-aa24-3bbd918a8813 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:13.737344 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7271ba99-15e8-4dda-aa24-3bbd918a8813 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:13.748892 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:13.763878 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f321337e-e7f8-4194-b46b-fc521e74b6ec None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:13.805021 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94627d6-2729-46a0-bcef-bbbc4ae8b3cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:13.805677 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94627d6-2729-46a0-bcef-bbbc4ae8b3cf None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:13.806604 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94627d6-2729-46a0-bcef-bbbc4ae8b3cf None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:13.806890 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94627d6-2729-46a0-bcef-bbbc4ae8b3cf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:13.824379 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94627d6-2729-46a0-bcef-bbbc4ae8b3cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:34:13.833758 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-f94627d6-2729-46a0-bcef-bbbc4ae8b3cf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 08:34:13.833758 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:13.833758 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:13.833758 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:13.833758 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:13.833758 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:13.833758 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:13.833758 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:13.833758 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:13.833758 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:13.833758 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:13.833758 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:13.833758 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:34:13.833758 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:34:13.833758 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:34:13.833758 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:34:13.834704 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:34:13.834704 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:34:13.834704 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:34:13.834704 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:34:13.834704 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:13.834704 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:13.834704 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:13.834704 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:13.834704 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:13.834704 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:13.834704 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:13.834704 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:13.834704 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:13.834704 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:13.834704 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:13.834704 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:13.835485 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:13.835485 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:13.835485 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:13.835485 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:13.835485 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:34:13.835485 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:34:13.835485 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:34:13.835485 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:34:13.835485 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:34:13.835485 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:34:13.835485 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 08:34:13.835485 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:34:13.835485 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 224/440] 10.222.0.55 () {56 vars in 1114 bytes} [Wed Aug 24 08:34:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:13.836877 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94627d6-2729-46a0-bcef-bbbc4ae8b3cf None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:13.910084 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 217/441] 10.222.0.55 () {58 vars in 1282 bytes} [Wed Aug 24 08:34:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:34:13.911610 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7271ba99-15e8-4dda-aa24-3bbd918a8813 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:14.009542 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94627d6-2729-46a0-bcef-bbbc4ae8b3cf None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:14.051605 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:14.057025 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7895f9-77d8-49e5-b976-e2844c20add2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:14.058420 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7895f9-77d8-49e5-b976-e2844c20add2 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:14.058966 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7895f9-77d8-49e5-b976-e2844c20add2 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:14.059397 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7895f9-77d8-49e5-b976-e2844c20add2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:14.078120 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7895f9-77d8-49e5-b976-e2844c20add2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:34:14.085204 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-3f7895f9-77d8-49e5-b976-e2844c20add2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:34:14.085204 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:14.085204 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:14.085204 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:14.085204 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:14.085204 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:14.085204 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:14.085204 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:14.085204 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:14.085204 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:14.085204 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:14.085204 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:14.085204 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:34:14.085204 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:34:14.085204 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:34:14.085204 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:34:14.087004 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:34:14.087004 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:34:14.087004 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:34:14.087004 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:34:14.087004 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:14.087004 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:14.087004 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:14.087004 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:14.087004 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:14.087004 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:14.087004 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:14.087004 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:14.087004 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:14.087004 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:14.087004 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:14.087004 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:14.088673 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:14.088673 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:14.088673 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:14.088673 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:14.088673 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:34:14.088673 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:34:14.088673 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:34:14.088673 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:34:14.088673 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:34:14.088673 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:34:14.088673 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:34:14.088673 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:34:14.088673 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 225/442] 10.222.0.55 () {56 vars in 1096 bytes} [Wed Aug 24 08:34:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:14.088673 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7895f9-77d8-49e5-b976-e2844c20add2 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:14.102990 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7271ba99-15e8-4dda-aa24-3bbd918a8813 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:14.106143 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:14.151336 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c0e75b-e7f0-4332-b5c6-b69db09d3e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:14.152002 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c0e75b-e7f0-4332-b5c6-b69db09d3e1b None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:14.152277 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c0e75b-e7f0-4332-b5c6-b69db09d3e1b None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:14.152569 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c0e75b-e7f0-4332-b5c6-b69db09d3e1b None admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:14.153260 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c0e75b-e7f0-4332-b5c6-b69db09d3e1b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:14.153561 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c0e75b-e7f0-4332-b5c6-b69db09d3e1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:14.154933 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c0e75b-e7f0-4332-b5c6-b69db09d3e1b None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:14.164015 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 218/443] 10.222.0.55 () {56 vars in 1111 bytes} [Wed Aug 24 08:34:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:14.164920 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c0e75b-e7f0-4332-b5c6-b69db09d3e1b None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:14.259089 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7895f9-77d8-49e5-b976-e2844c20add2 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:14.308801 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abcc723-e31b-43db-b575-b2d6c11d2b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:14.309596 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abcc723-e31b-43db-b575-b2d6c11d2b27 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:14.309939 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abcc723-e31b-43db-b575-b2d6c11d2b27 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:14.310279 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abcc723-e31b-43db-b575-b2d6c11d2b27 None admin] PATH_INFO: `/v3/projects/61247a1049b846b79052f42b0d525259` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:14.310707 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c0e75b-e7f0-4332-b5c6-b69db09d3e1b None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:14.311758 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abcc723-e31b-43db-b575-b2d6c11d2b27 None admin] RBAC: Authorizing `identity:get_project(project_id=61247a1049b846b79052f42b0d525259)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:14.313574 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abcc723-e31b-43db-b575-b2d6c11d2b27 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:14.315473 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 226/444] 10.222.0.55 () {56 vars in 1186 bytes} [Wed Aug 24 08:34:14 2022] GET /identity/v3/projects/61247a1049b846b79052f42b0d525259 => generated 302 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:14.316411 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abcc723-e31b-43db-b575-b2d6c11d2b27 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:14.354353 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dd39cb-d6d1-4c8e-8ca9-0ff76d2e221f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:14.355434 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dd39cb-d6d1-4c8e-8ca9-0ff76d2e221f None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:14.355707 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dd39cb-d6d1-4c8e-8ca9-0ff76d2e221f None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:14.355976 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dd39cb-d6d1-4c8e-8ca9-0ff76d2e221f None admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:14.356677 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dd39cb-d6d1-4c8e-8ca9-0ff76d2e221f None admin] RBAC: Adding query filter params (name=admin) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:14.356979 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dd39cb-d6d1-4c8e-8ca9-0ff76d2e221f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:14.358354 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dd39cb-d6d1-4c8e-8ca9-0ff76d2e221f None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:14.368282 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 219/445] 10.222.0.55 () {56 vars in 1099 bytes} [Wed Aug 24 08:34:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:14.369172 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dd39cb-d6d1-4c8e-8ca9-0ff76d2e221f None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:14.372774 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:14.424755 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:14.459209 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abcc723-e31b-43db-b575-b2d6c11d2b27 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:14.495627 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e609c945-6e7e-405d-a8eb-900d91270b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:14.496383 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e609c945-6e7e-405d-a8eb-900d91270b5a None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:14.496787 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e609c945-6e7e-405d-a8eb-900d91270b5a None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:14.497145 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e609c945-6e7e-405d-a8eb-900d91270b5a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:14.511322 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dd39cb-d6d1-4c8e-8ca9-0ff76d2e221f None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:14.572416 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2cfabf-feb3-4885-befa-000f7ae08715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:14.573956 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2cfabf-feb3-4885-befa-000f7ae08715 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:14.574447 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2cfabf-feb3-4885-befa-000f7ae08715 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:14.574928 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2cfabf-feb3-4885-befa-000f7ae08715 None admin] PATH_INFO: `/v3/groups/121aa5cb19db48029d2846f8b2bf90de` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:14.577039 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2cfabf-feb3-4885-befa-000f7ae08715 None admin] RBAC: Authorizing `identity:get_group(group_id=121aa5cb19db48029d2846f8b2bf90de)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:14.579692 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2cfabf-feb3-4885-befa-000f7ae08715 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:14.581375 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e609c945-6e7e-405d-a8eb-900d91270b5a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:14.588904 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 220/446] 10.222.0.55 () {56 vars in 1180 bytes} [Wed Aug 24 08:34:14 2022] GET /identity/v3/groups/121aa5cb19db48029d2846f8b2bf90de => generated 226 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:14.589808 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e609c945-6e7e-405d-a8eb-900d91270b5a None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:14.589978 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2cfabf-feb3-4885-befa-000f7ae08715 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:14.671311 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:14.710275 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:14.712845 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2cfabf-feb3-4885-befa-000f7ae08715 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:14.741111 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 227/447] 10.222.0.55 () {58 vars in 1282 bytes} [Wed Aug 24 08:34:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:34:14.742803 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e609c945-6e7e-405d-a8eb-900d91270b5a None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:14.756323 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa584d5-14d3-455d-b812-e9b598b8df14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:14.757048 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa584d5-14d3-455d-b812-e9b598b8df14 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:14.757384 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa584d5-14d3-455d-b812-e9b598b8df14 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:14.757715 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa584d5-14d3-455d-b812-e9b598b8df14 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:14.838201 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa584d5-14d3-455d-b812-e9b598b8df14 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:14.840311 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa584d5-14d3-455d-b812-e9b598b8df14 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:14.882999 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e609c945-6e7e-405d-a8eb-900d91270b5a None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:14.921013 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e072ad-31f6-4a2c-a75f-1394668bc861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:14.921685 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e072ad-31f6-4a2c-a75f-1394668bc861 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:14.921912 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e072ad-31f6-4a2c-a75f-1394668bc861 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:14.922187 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e072ad-31f6-4a2c-a75f-1394668bc861 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:14.922932 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e072ad-31f6-4a2c-a75f-1394668bc861 None admin] RBAC: Adding query filter params (group.id=121aa5cb19db48029d2846f8b2bf90de, role.id=b674fd0a592e4e0cada811bba5ceea4d, scope.project.id=61247a1049b846b79052f42b0d525259) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:14.923239 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e072ad-31f6-4a2c-a75f-1394668bc861 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:14.924741 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e072ad-31f6-4a2c-a75f-1394668bc861 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:14.925453 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:14.931669 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 228/448] 10.222.0.55 () {56 vars in 1376 bytes} [Wed Aug 24 08:34:14 2022] GET /identity/v3/role_assignments?group.id=121aa5cb19db48029d2846f8b2bf90de&scope.project.id=61247a1049b846b79052f42b0d525259&role.id=b674fd0a592e4e0cada811bba5ceea4d => generated 611 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:14.932588 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e072ad-31f6-4a2c-a75f-1394668bc861 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:14.971551 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 221/449] 10.222.0.55 () {58 vars in 1282 bytes} [Wed Aug 24 08:34:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:34:14.975032 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa584d5-14d3-455d-b812-e9b598b8df14 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:15.007282 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:15.119894 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa584d5-14d3-455d-b812-e9b598b8df14 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:15.128486 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e072ad-31f6-4a2c-a75f-1394668bc861 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:15.184411 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772b39dd-63ae-483e-aeaa-742d975dfb87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:15.185092 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772b39dd-63ae-483e-aeaa-742d975dfb87 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:15.185385 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772b39dd-63ae-483e-aeaa-742d975dfb87 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:15.185716 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772b39dd-63ae-483e-aeaa-742d975dfb87 None admin] PATH_INFO: `/v3/projects/61247a1049b846b79052f42b0d525259` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:15.187367 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1407b0a2-999c-48a8-86dc-3122c0c583da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:15.187500 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772b39dd-63ae-483e-aeaa-742d975dfb87 None admin] RBAC: Authorizing `identity:get_project(project_id=61247a1049b846b79052f42b0d525259)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:15.187573 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1407b0a2-999c-48a8-86dc-3122c0c583da None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:15.188504 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1407b0a2-999c-48a8-86dc-3122c0c583da None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:15.188504 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1407b0a2-999c-48a8-86dc-3122c0c583da None admin] PATH_INFO: `/v3/projects/6ad87fa1813247aba83e93f886f34eb3` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:15.188900 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772b39dd-63ae-483e-aeaa-742d975dfb87 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:15.190523 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1407b0a2-999c-48a8-86dc-3122c0c583da None admin] RBAC: Authorizing `identity:get_project(project_id=6ad87fa1813247aba83e93f886f34eb3)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:15.191451 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 229/450] 10.222.0.55 () {56 vars in 1186 bytes} [Wed Aug 24 08:34:14 2022] GET /identity/v3/projects/61247a1049b846b79052f42b0d525259 => generated 302 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:15.194169 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1407b0a2-999c-48a8-86dc-3122c0c583da None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:15.196790 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 222/451] 10.222.0.55 () {56 vars in 1186 bytes} [Wed Aug 24 08:34:14 2022] GET /identity/v3/projects/6ad87fa1813247aba83e93f886f34eb3 => generated 344 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:34:15.209074 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772b39dd-63ae-483e-aeaa-742d975dfb87 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:15.209627 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1407b0a2-999c-48a8-86dc-3122c0c583da None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:15.269912 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:15.372989 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1407b0a2-999c-48a8-86dc-3122c0c583da None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:15.400765 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772b39dd-63ae-483e-aeaa-742d975dfb87 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:15.411402 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:15.426294 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493b847f-ab7b-4266-b680-a9184a10b8da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:15.426945 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493b847f-ab7b-4266-b680-a9184a10b8da None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:15.427215 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493b847f-ab7b-4266-b680-a9184a10b8da None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:15.427482 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493b847f-ab7b-4266-b680-a9184a10b8da None admin] PATH_INFO: `/v3/groups/121aa5cb19db48029d2846f8b2bf90de` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:15.428765 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493b847f-ab7b-4266-b680-a9184a10b8da None admin] RBAC: Authorizing `identity:get_group(group_id=121aa5cb19db48029d2846f8b2bf90de)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:15.430242 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493b847f-ab7b-4266-b680-a9184a10b8da None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:15.431938 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 223/452] 10.222.0.55 () {56 vars in 1180 bytes} [Wed Aug 24 08:34:15 2022] GET /identity/v3/groups/121aa5cb19db48029d2846f8b2bf90de => generated 226 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:34:15.439272 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493b847f-ab7b-4266-b680-a9184a10b8da None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:15.448393 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9c87b4-7362-4cc4-bc0c-7f503d9f9127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:15.449391 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9c87b4-7362-4cc4-bc0c-7f503d9f9127 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:15.449669 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9c87b4-7362-4cc4-bc0c-7f503d9f9127 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:15.449942 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9c87b4-7362-4cc4-bc0c-7f503d9f9127 None admin] PATH_INFO: `/v3/groups/a37059af981c4b8b8bee1a21f0f63ebb` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:15.451510 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9c87b4-7362-4cc4-bc0c-7f503d9f9127 None admin] RBAC: Authorizing `identity:get_group(group_id=a37059af981c4b8b8bee1a21f0f63ebb)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:15.453087 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9c87b4-7362-4cc4-bc0c-7f503d9f9127 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:15.454819 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 230/453] 10.222.0.55 () {56 vars in 1180 bytes} [Wed Aug 24 08:34:15 2022] GET /identity/v3/groups/a37059af981c4b8b8bee1a21f0f63ebb => generated 229 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:34:15.461192 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9c87b4-7362-4cc4-bc0c-7f503d9f9127 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:15.568494 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493b847f-ab7b-4266-b680-a9184a10b8da None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:15.580443 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:15.610805 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5fae98-8097-4400-80e2-4a3063f7bbb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:15.611420 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5fae98-8097-4400-80e2-4a3063f7bbb2 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:15.611693 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5fae98-8097-4400-80e2-4a3063f7bbb2 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:15.611962 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5fae98-8097-4400-80e2-4a3063f7bbb2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:15.612721 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5fae98-8097-4400-80e2-4a3063f7bbb2 None admin] RBAC: Adding query filter params (group.id=121aa5cb19db48029d2846f8b2bf90de, role.id=8b5ad96041894155b0138a676ead607e, scope.project.id=61247a1049b846b79052f42b0d525259) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:15.613026 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5fae98-8097-4400-80e2-4a3063f7bbb2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:15.614374 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9c87b4-7362-4cc4-bc0c-7f503d9f9127 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:15.614515 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5fae98-8097-4400-80e2-4a3063f7bbb2 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:15.621616 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 224/454] 10.222.0.55 () {56 vars in 1376 bytes} [Wed Aug 24 08:34:15 2022] GET /identity/v3/role_assignments?group.id=121aa5cb19db48029d2846f8b2bf90de&scope.project.id=61247a1049b846b79052f42b0d525259&role.id=8b5ad96041894155b0138a676ead607e => generated 611 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:34:15.679575 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3d7681-5deb-4d16-a8d6-dd68536f619b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:15.680750 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d7681-5deb-4d16-a8d6-dd68536f619b None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:15.681246 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d7681-5deb-4d16-a8d6-dd68536f619b None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:15.681727 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d7681-5deb-4d16-a8d6-dd68536f619b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:15.683705 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3d7681-5deb-4d16-a8d6-dd68536f619b None admin] RBAC: Adding query filter params (group.id=a37059af981c4b8b8bee1a21f0f63ebb, role.id=b870cd1acaa744e1ae1d58b9ac283821, scope.project.id=6ad87fa1813247aba83e93f886f34eb3) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:15.684112 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3d7681-5deb-4d16-a8d6-dd68536f619b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:15.686867 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3d7681-5deb-4d16-a8d6-dd68536f619b None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:15.696673 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 231/455] 10.222.0.55 () {56 vars in 1376 bytes} [Wed Aug 24 08:34:15 2022] GET /identity/v3/role_assignments?group.id=a37059af981c4b8b8bee1a21f0f63ebb&scope.project.id=6ad87fa1813247aba83e93f886f34eb3&role.id=b870cd1acaa744e1ae1d58b9ac283821 => generated 611 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:34:18.272753 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ee89d1-c998-4dda-9cac-a23f92baa6b0 None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:18.273530 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ee89d1-c998-4dda-9cac-a23f92baa6b0 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:18.273710 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ee89d1-c998-4dda-9cac-a23f92baa6b0 None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:18.276465 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 225/456] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:34:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:34:18.283510 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29164e2-e672-4ba5-82aa-4588a2b011e1 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:18.284889 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29164e2-e672-4ba5-82aa-4588a2b011e1 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:18.285601 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29164e2-e672-4ba5-82aa-4588a2b011e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:18.330454 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-c29164e2-e672-4ba5-82aa-4588a2b011e1 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:18.332856 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-c29164e2-e672-4ba5-82aa-4588a2b011e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:18.423162 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:18.504812 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 232/457] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 08:34:18.930446 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ee89d1-c998-4dda-9cac-a23f92baa6b0 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:19.101919 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ee89d1-c998-4dda-9cac-a23f92baa6b0 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:19.155464 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4258451d-5e04-4d13-8a98-687725fa3ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:19.156131 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4258451d-5e04-4d13-8a98-687725fa3ffd None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:19.156412 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4258451d-5e04-4d13-8a98-687725fa3ffd None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:19.156720 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4258451d-5e04-4d13-8a98-687725fa3ffd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:19.173923 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4258451d-5e04-4d13-8a98-687725fa3ffd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:19.175377 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4258451d-5e04-4d13-8a98-687725fa3ffd None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:19.194234 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-4258451d-5e04-4d13-8a98-687725fa3ffd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:34:19.194234 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:19.194234 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:19.194234 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:19.194234 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:19.194234 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:19.194234 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:19.194234 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:19.194234 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:19.194234 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:19.194234 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:19.194234 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:19.194234 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:34:19.194234 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:34:19.194234 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:34:19.194234 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:34:19.195191 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:19.195191 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:19.195191 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:19.195191 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:19.195191 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:19.195191 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:19.195191 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:19.195191 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:19.195191 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:19.195191 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:19.195191 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:19.195191 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:19.195191 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:19.195191 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:19.195191 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:19.195191 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:19.195191 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:34:19.196053 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:34:19.196053 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:34:19.196053 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:34:19.196675 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 226/458] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:34:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:34:19.210396 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29164e2-e672-4ba5-82aa-4588a2b011e1 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:19.257508 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9e3145-7dcd-4d83-94a3-b0ceee99f2d9 None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:19.258151 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9e3145-7dcd-4d83-94a3-b0ceee99f2d9 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:19.258628 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9e3145-7dcd-4d83-94a3-b0ceee99f2d9 None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:19.260601 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 227/459] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:34:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:34:19.266576 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa743b4-78ab-45f8-a7d7-c6e8b05543ad None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:19.267159 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa743b4-78ab-45f8-a7d7-c6e8b05543ad None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:19.267628 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa743b4-78ab-45f8-a7d7-c6e8b05543ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:19.309846 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-7fa743b4-78ab-45f8-a7d7-c6e8b05543ad None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:19.311917 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-7fa743b4-78ab-45f8-a7d7-c6e8b05543ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:19.351839 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:19.372186 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29164e2-e672-4ba5-82aa-4588a2b011e1 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:19.428622 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dccf9a-c0e3-43ed-9323-1a7d4e0474d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:19.429538 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dccf9a-c0e3-43ed-9323-1a7d4e0474d3 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:19.429988 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dccf9a-c0e3-43ed-9323-1a7d4e0474d3 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:19.430409 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dccf9a-c0e3-43ed-9323-1a7d4e0474d3 None admin] PATH_INFO: `/v3/domains` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:19.431234 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dccf9a-c0e3-43ed-9323-1a7d4e0474d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:19.431686 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dccf9a-c0e3-43ed-9323-1a7d4e0474d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:19.433909 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dccf9a-c0e3-43ed-9323-1a7d4e0474d3 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:19.459891 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 233/460] 10.222.0.55 () {56 vars in 1109 bytes} [Wed Aug 24 08:34:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:19.465832 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dccf9a-c0e3-43ed-9323-1a7d4e0474d3 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:19.513587 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:19.517453 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 228/461] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:34:19.595776 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dccf9a-c0e3-43ed-9323-1a7d4e0474d3 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:19.640574 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9849f3-83c0-4145-a669-d7a8421e36b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:19.641460 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9849f3-83c0-4145-a669-d7a8421e36b5 None admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:19.641891 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9849f3-83c0-4145-a669-d7a8421e36b5 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:19.642310 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9849f3-83c0-4145-a669-d7a8421e36b5 None admin] PATH_INFO: `/v3/users` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:19.643143 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9849f3-83c0-4145-a669-d7a8421e36b5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:19.644769 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9849f3-83c0-4145-a669-d7a8421e36b5 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:19.711178 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8831ef-6239-4f18-81b6-28e58352dd03 None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:19.711678 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8831ef-6239-4f18-81b6-28e58352dd03 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:19.712099 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8831ef-6239-4f18-81b6-28e58352dd03 None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:19.713906 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 229/462] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:34:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:34:19.719662 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3bbd03-d996-4218-894b-3eb6e95040eb None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:19.720142 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3bbd03-d996-4218-894b-3eb6e95040eb None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:19.720615 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3bbd03-d996-4218-894b-3eb6e95040eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:19.741615 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-4a3bbd03-d996-4218-894b-3eb6e95040eb None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:19.743112 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-4a3bbd03-d996-4218-894b-3eb6e95040eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:19.772302 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 234/463] 10.222.0.55 () {60 vars in 1132 bytes} [Wed Aug 24 08:34:19 2022] POST /identity/v3/users => generated 246 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:34:19.858405 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9849f3-83c0-4145-a669-d7a8421e36b5 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:19.859267 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 230/464] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:34:20.009258 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:20.032572 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9849f3-83c0-4145-a669-d7a8421e36b5 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:20.081770 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d177e27c-c970-4222-81de-776bd8f74fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:20.082426 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d177e27c-c970-4222-81de-776bd8f74fa3 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:20.082706 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d177e27c-c970-4222-81de-776bd8f74fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:20.082980 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d177e27c-c970-4222-81de-776bd8f74fa3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:20.101700 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d177e27c-c970-4222-81de-776bd8f74fa3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:20.103193 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d177e27c-c970-4222-81de-776bd8f74fa3 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:20.108191 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeb179d-2ebb-4ae8-a180-ac1a1a2d0dde None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:20.108627 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeb179d-2ebb-4ae8-a180-ac1a1a2d0dde None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:20.108776 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeb179d-2ebb-4ae8-a180-ac1a1a2d0dde None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:20.110599 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 231/465] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:34:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:34:20.123789 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcaea2c-06d6-45db-b0d7-245ae14d9875 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:20.125029 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcaea2c-06d6-45db-b0d7-245ae14d9875 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:20.125527 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcaea2c-06d6-45db-b0d7-245ae14d9875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:20.134339 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-d177e27c-c970-4222-81de-776bd8f74fa3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:34:20.134339 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:20.134339 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:20.134339 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:20.134339 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:20.134339 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:20.134339 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:20.134339 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:20.134339 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:20.134339 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:20.134339 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:20.134339 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:20.134339 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:34:20.134339 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:34:20.134339 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:34:20.134339 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:34:20.135329 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:20.135329 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:20.135329 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:20.135329 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:20.135329 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:20.135329 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:20.135329 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:20.135329 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:20.135329 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:20.135329 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:20.135329 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:20.135329 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:20.135329 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:20.135329 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:20.135329 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:20.135329 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:20.135329 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:34:20.136210 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:34:20.136210 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:34:20.136210 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:34:20.139563 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 235/466] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:34:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:34:20.141882 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d177e27c-c970-4222-81de-776bd8f74fa3 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:20.150016 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-8fcaea2c-06d6-45db-b0d7-245ae14d9875 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:20.151593 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-8fcaea2c-06d6-45db-b0d7-245ae14d9875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:20.270014 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 232/467] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 08:34:20.287059 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d177e27c-c970-4222-81de-776bd8f74fa3 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:20.294492 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcaea2c-06d6-45db-b0d7-245ae14d9875 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:20.324023 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:20.352820 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1af6a0f-ee45-45cd-8c7b-5545f77cc02c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:20.353445 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1af6a0f-ee45-45cd-8c7b-5545f77cc02c None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:20.353947 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1af6a0f-ee45-45cd-8c7b-5545f77cc02c None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:20.354374 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1af6a0f-ee45-45cd-8c7b-5545f77cc02c None admin] PATH_INFO: `/v3/domains` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:20.355751 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1af6a0f-ee45-45cd-8c7b-5545f77cc02c None admin] RBAC: Adding query filter params (name=Default) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:20.356221 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1af6a0f-ee45-45cd-8c7b-5545f77cc02c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:20.357868 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1af6a0f-ee45-45cd-8c7b-5545f77cc02c None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:20.367205 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:20.374192 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 236/468] 10.222.0.55 () {56 vars in 1109 bytes} [Wed Aug 24 08:34:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:34:20.380877 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1af6a0f-ee45-45cd-8c7b-5545f77cc02c None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:20.498031 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1af6a0f-ee45-45cd-8c7b-5545f77cc02c None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:20.515088 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcaea2c-06d6-45db-b0d7-245ae14d9875 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:20.541778 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118f956a-e46b-4823-a43c-db4c833f2a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:20.543078 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118f956a-e46b-4823-a43c-db4c833f2a69 None admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:20.543388 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118f956a-e46b-4823-a43c-db4c833f2a69 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:20.543762 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118f956a-e46b-4823-a43c-db4c833f2a69 None admin] PATH_INFO: `/v3/users` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:20.544580 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118f956a-e46b-4823-a43c-db4c833f2a69 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:20.546387 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118f956a-e46b-4823-a43c-db4c833f2a69 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:20.581495 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e77972-ad98-4c12-9ddb-ebb2a381d81c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:20.582598 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e77972-ad98-4c12-9ddb-ebb2a381d81c None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:20.583117 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e77972-ad98-4c12-9ddb-ebb2a381d81c None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:20.583636 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e77972-ad98-4c12-9ddb-ebb2a381d81c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:20.597207 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e77972-ad98-4c12-9ddb-ebb2a381d81c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:20.599069 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e77972-ad98-4c12-9ddb-ebb2a381d81c None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:20.616797 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-67e77972-ad98-4c12-9ddb-ebb2a381d81c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:34:20.616797 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:20.616797 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:20.616797 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:20.616797 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:20.616797 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:20.616797 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:20.616797 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:20.616797 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:20.616797 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:20.616797 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:20.616797 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:20.616797 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:34:20.616797 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:34:20.616797 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:34:20.616797 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:34:20.617813 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:20.617813 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:20.617813 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:20.617813 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:20.617813 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:20.617813 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:20.617813 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:20.617813 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:20.617813 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:20.617813 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:20.617813 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:20.617813 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:20.617813 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:20.617813 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:20.617813 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:20.617813 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:20.617813 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:34:20.618670 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:34:20.618670 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:34:20.618670 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:34:20.620911 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 233/469] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:34:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:34:20.630424 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e77972-ad98-4c12-9ddb-ebb2a381d81c None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:20.653823 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 237/470] 10.222.0.55 () {60 vars in 1132 bytes} [Wed Aug 24 08:34:20 2022] POST /identity/v3/users => generated 248 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:34:20.691006 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118f956a-e46b-4823-a43c-db4c833f2a69 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:20.724364 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:20.771213 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e77972-ad98-4c12-9ddb-ebb2a381d81c None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:20.810773 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:20.811919 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e992ac79-e417-4b3e-90ea-af4b18a77ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:20.812795 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e992ac79-e417-4b3e-90ea-af4b18a77ffb None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:20.813234 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e992ac79-e417-4b3e-90ea-af4b18a77ffb None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:20.813655 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e992ac79-e417-4b3e-90ea-af4b18a77ffb None admin] PATH_INFO: `/v3/domains` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:20.814488 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e992ac79-e417-4b3e-90ea-af4b18a77ffb None admin] RBAC: Adding query filter params (name=Default) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:20.814940 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e992ac79-e417-4b3e-90ea-af4b18a77ffb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:20.816583 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e992ac79-e417-4b3e-90ea-af4b18a77ffb None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:20.832209 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 234/471] 10.222.0.55 () {56 vars in 1109 bytes} [Wed Aug 24 08:34:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:20.857552 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e992ac79-e417-4b3e-90ea-af4b18a77ffb None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:20.919563 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118f956a-e46b-4823-a43c-db4c833f2a69 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:20.960461 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5637cd81-a0be-406a-9a7e-4494364a2a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:20.960942 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5637cd81-a0be-406a-9a7e-4494364a2a81 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:20.961339 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5637cd81-a0be-406a-9a7e-4494364a2a81 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:20.961706 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5637cd81-a0be-406a-9a7e-4494364a2a81 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:20.974213 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5637cd81-a0be-406a-9a7e-4494364a2a81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:20.975848 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5637cd81-a0be-406a-9a7e-4494364a2a81 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:20.995130 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-5637cd81-a0be-406a-9a7e-4494364a2a81 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:34:20.995130 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:20.995130 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:20.995130 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:20.995130 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:20.995130 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:20.995130 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:20.995130 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:20.995130 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:20.995130 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:20.995130 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:20.995130 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:20.995130 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:34:20.995130 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:34:20.995130 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:34:20.995130 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:34:20.997408 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:20.997408 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:20.997408 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:20.997408 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:20.997408 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:20.997408 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:20.997408 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:20.997408 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:20.997408 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:20.997408 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:20.997408 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:20.997408 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:20.997408 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:20.997408 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:20.997408 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:20.997408 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:20.997408 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:34:20.999092 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:34:20.999092 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:34:20.999092 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:34:20.999092 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 238/472] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:34:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:20.999092 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105905c4-2d6e-4d32-a0c4-6d8220ba0bae None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:20.999092 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105905c4-2d6e-4d32-a0c4-6d8220ba0bae None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:21.002658 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105905c4-2d6e-4d32-a0c4-6d8220ba0bae None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:21.002658 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 239/473] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:34:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:34:21.006354 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105905c4-2d6e-4d32-a0c4-6d8220ba0bae None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:21.037278 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e992ac79-e417-4b3e-90ea-af4b18a77ffb None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:21.081782 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:21.099014 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e721b1-9bde-4e46-b96d-49a126fda090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:21.100527 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e721b1-9bde-4e46-b96d-49a126fda090 None admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:21.100828 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e721b1-9bde-4e46-b96d-49a126fda090 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:21.101407 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e721b1-9bde-4e46-b96d-49a126fda090 None admin] PATH_INFO: `/v3/users` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:21.102694 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e721b1-9bde-4e46-b96d-49a126fda090 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:21.105671 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e721b1-9bde-4e46-b96d-49a126fda090 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:21.223824 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 235/474] 10.222.0.55 () {60 vars in 1132 bytes} [Wed Aug 24 08:34:20 2022] POST /identity/v3/users => generated 249 bytes in 367 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:34:21.227870 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b1d313-3586-4e9b-926d-18ca2f47f721 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:21.229330 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b1d313-3586-4e9b-926d-18ca2f47f721 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:21.230205 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b1d313-3586-4e9b-926d-18ca2f47f721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:21.234098 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105905c4-2d6e-4d32-a0c4-6d8220ba0bae None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:21.251019 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:21.273883 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-12b1d313-3586-4e9b-926d-18ca2f47f721 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:21.277940 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-12b1d313-3586-4e9b-926d-18ca2f47f721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:21.372932 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062b0c25-28ef-4159-b11a-da34a07db9c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:21.374008 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062b0c25-28ef-4159-b11a-da34a07db9c8 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:21.374498 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062b0c25-28ef-4159-b11a-da34a07db9c8 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:21.375354 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062b0c25-28ef-4159-b11a-da34a07db9c8 None admin] PATH_INFO: `/v3/domains` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:21.376385 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062b0c25-28ef-4159-b11a-da34a07db9c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:21.377218 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062b0c25-28ef-4159-b11a-da34a07db9c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:21.379976 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062b0c25-28ef-4159-b11a-da34a07db9c8 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:21.414080 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 240/475] 10.222.0.55 () {56 vars in 1109 bytes} [Wed Aug 24 08:34:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 409 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:21.420355 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062b0c25-28ef-4159-b11a-da34a07db9c8 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:21.443578 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 236/476] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:34:21.635090 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062b0c25-28ef-4159-b11a-da34a07db9c8 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:21.706368 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c354e3-fbc0-4bd0-ae0c-b2ab52c9546c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:21.707838 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c354e3-fbc0-4bd0-ae0c-b2ab52c9546c None admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:21.708576 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c354e3-fbc0-4bd0-ae0c-b2ab52c9546c None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:21.709401 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c354e3-fbc0-4bd0-ae0c-b2ab52c9546c None admin] PATH_INFO: `/v3/users` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:21.710704 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c354e3-fbc0-4bd0-ae0c-b2ab52c9546c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:21.713765 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c354e3-fbc0-4bd0-ae0c-b2ab52c9546c None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:21.810249 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 241/477] 10.222.0.55 () {60 vars in 1132 bytes} [Wed Aug 24 08:34:21 2022] POST /identity/v3/users => generated 248 bytes in 391 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:34:21.881120 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b1d313-3586-4e9b-926d-18ca2f47f721 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:22.085410 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b1d313-3586-4e9b-926d-18ca2f47f721 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:22.117890 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:22.149828 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e1233f-f1b3-4205-b458-f711ff41665f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:22.150908 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e1233f-f1b3-4205-b458-f711ff41665f None admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:22.151254 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e1233f-f1b3-4205-b458-f711ff41665f None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:22.151538 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e1233f-f1b3-4205-b458-f711ff41665f None admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:22.153514 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e1233f-f1b3-4205-b458-f711ff41665f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:22.156334 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e1233f-f1b3-4205-b458-f711ff41665f None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:22.216317 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.sql.core [None req-b7e1233f-f1b3-4205-b458-f711ff41665f None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 08:34:22.216317 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 24 08:34:22.216317 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [parameters: {'id': '55193225f21b4d86a945c1290ff578e6', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 24 08:34:22.216317 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=93542) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 08:34:22.248235 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-b7e1233f-f1b3-4205-b458-f711ff41665f None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 24 08:34:22.248235 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:22.248235 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 08:34:22.248235 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 08:34:22.248235 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 08:34:22.248235 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 08:34:22.248235 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 08:34:22.248235 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 08:34:22.248235 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 08:34:22.248235 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application conn.query(q) Aug 24 08:34:22.248235 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 08:34:22.248235 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 08:34:22.248235 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 08:34:22.248235 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application result.read() Aug 24 08:34:22.248235 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 08:34:22.249229 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 08:34:22.249229 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 08:34:22.249229 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 08:34:22.249229 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 08:34:22.249229 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 08:34:22.249229 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 08:34:22.249229 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 08:34:22.249229 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 08:34:22.249229 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:34:22.249229 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 08:34:22.249229 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:34:22.249229 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:22.249229 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 08:34:22.249229 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 08:34:22.249229 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 24 08:34:22.249229 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return ref.to_dict() Aug 24 08:34:22.249229 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 08:34:22.249229 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application next(self.gen) Aug 24 08:34:22.250143 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 08:34:22.250143 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application yield resource Aug 24 08:34:22.250143 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 08:34:22.250143 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application next(self.gen) Aug 24 08:34:22.250143 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 08:34:22.250143 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 08:34:22.250143 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 08:34:22.250143 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 08:34:22.250143 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 08:34:22.250143 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise self.value Aug 24 08:34:22.250143 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 08:34:22.250143 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 08:34:22.250143 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 08:34:22.250143 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application session.commit() Aug 24 08:34:22.250143 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 08:34:22.250143 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 08:34:22.250143 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 08:34:22.251019 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 08:34:22.251019 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 08:34:22.251019 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application self.session.flush() Aug 24 08:34:22.251019 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 08:34:22.251019 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 08:34:22.251019 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 08:34:22.251019 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 08:34:22.251019 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 08:34:22.251019 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application compat.raise_( Aug 24 08:34:22.251019 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 08:34:22.251019 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception Aug 24 08:34:22.251019 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 08:34:22.251019 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 08:34:22.251019 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 08:34:22.251019 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 08:34:22.251019 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 08:34:22.251019 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 08:34:22.252181 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 08:34:22.252181 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 08:34:22.252181 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 24 08:34:22.252181 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 24 08:34:22.252181 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 08:34:22.252181 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 08:34:22.252181 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 08:34:22.252181 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 08:34:22.252181 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 08:34:22.252181 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 08:34:22.252181 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 08:34:22.252181 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 08:34:22.252181 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 08:34:22.252181 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 08:34:22.252181 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 08:34:22.252181 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception Aug 24 08:34:22.254580 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 08:34:22.254580 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 08:34:22.254580 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 08:34:22.254580 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 08:34:22.254580 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 08:34:22.254580 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 08:34:22.254580 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 08:34:22.254580 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application conn.query(q) Aug 24 08:34:22.254580 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 08:34:22.254580 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 08:34:22.254580 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 08:34:22.254580 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application result.read() Aug 24 08:34:22.254580 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 08:34:22.254580 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 08:34:22.254580 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 08:34:22.254580 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 08:34:22.256658 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 08:34:22.256658 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 08:34:22.256658 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 08:34:22.256658 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 08:34:22.256658 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 08:34:22.256658 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 24 08:34:22.256658 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [parameters: {'id': '55193225f21b4d86a945c1290ff578e6', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 24 08:34:22.256658 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 08:34:22.256658 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:34:22.256658 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:34:22.256658 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:34:22.256658 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:22.256658 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:22.256658 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:22.256658 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:22.256658 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:22.258689 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:22.258689 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:22.258689 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:22.258689 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:22.258689 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:22.258689 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:22.258689 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 24 08:34:22.258689 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 24 08:34:22.258689 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:22.258689 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:22.258689 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 24 08:34:22.258689 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 24 08:34:22.258689 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 08:34:22.258689 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 08:34:22.258689 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 24 08:34:22.258689 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:34:22.263596 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 237/478] 10.222.0.55 () {60 vars in 1131 bytes} [Wed Aug 24 08:34:21 2022] POST /identity/v3/roles => generated 144 bytes in 370 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:22.269455 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c354e3-fbc0-4bd0-ae0c-b2ab52c9546c None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:22.299644 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:22.395046 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c354e3-fbc0-4bd0-ae0c-b2ab52c9546c None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:22.436154 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c3efdb-f458-4c52-a9ae-3abbc3a5f900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:22.437180 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c3efdb-f458-4c52-a9ae-3abbc3a5f900 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:22.437753 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c3efdb-f458-4c52-a9ae-3abbc3a5f900 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:22.438242 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c3efdb-f458-4c52-a9ae-3abbc3a5f900 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:22.451472 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c3efdb-f458-4c52-a9ae-3abbc3a5f900 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:34:22.457249 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-21c3efdb-f458-4c52-a9ae-3abbc3a5f900 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 08:34:22.457249 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:22.457249 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:22.457249 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:22.457249 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:22.457249 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:22.457249 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:22.457249 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:22.457249 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:22.457249 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:22.457249 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:22.457249 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:22.457249 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:34:22.457249 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:34:22.457249 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:34:22.457249 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:34:22.458202 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:34:22.458202 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:34:22.458202 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:34:22.458202 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:34:22.458202 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:22.458202 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:22.458202 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:22.458202 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:22.458202 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:22.458202 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:22.458202 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:22.458202 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:22.458202 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:22.458202 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:22.458202 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:22.458202 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:22.458989 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:22.458989 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:22.458989 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:22.458989 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:22.458989 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:34:22.458989 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:34:22.458989 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:34:22.458989 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:34:22.458989 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:34:22.458989 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:34:22.458989 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 08:34:22.458989 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:34:22.461746 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 242/479] 10.222.0.55 () {56 vars in 1114 bytes} [Wed Aug 24 08:34:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:22.467870 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e1233f-f1b3-4205-b458-f711ff41665f None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:22.594814 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e1233f-f1b3-4205-b458-f711ff41665f None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:22.649469 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0282275b-9e9a-4697-8a35-f20123ec4352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:22.650028 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0282275b-9e9a-4697-8a35-f20123ec4352 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:22.650311 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0282275b-9e9a-4697-8a35-f20123ec4352 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:22.650588 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0282275b-9e9a-4697-8a35-f20123ec4352 None admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:22.651278 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0282275b-9e9a-4697-8a35-f20123ec4352 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:22.651581 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0282275b-9e9a-4697-8a35-f20123ec4352 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:22.653064 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0282275b-9e9a-4697-8a35-f20123ec4352 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:22.662630 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 238/480] 10.222.0.55 () {56 vars in 1111 bytes} [Wed Aug 24 08:34:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:22.786467 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0397861-acbc-49cf-9441-49aac86bf597 None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:22.787159 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0397861-acbc-49cf-9441-49aac86bf597 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:22.787637 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0397861-acbc-49cf-9441-49aac86bf597 None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:22.789631 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 243/481] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:34:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:34:22.795251 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561ebe8f-7239-4329-8415-c68afdf2d78f None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:22.795632 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561ebe8f-7239-4329-8415-c68afdf2d78f None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:22.795828 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561ebe8f-7239-4329-8415-c68afdf2d78f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:22.823266 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-561ebe8f-7239-4329-8415-c68afdf2d78f None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:22.825101 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-561ebe8f-7239-4329-8415-c68afdf2d78f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:22.860647 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:22.911575 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 239/482] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:34:23.139890 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be326243-3d35-4d35-9f66-43ac5032e31b None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:23.140280 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be326243-3d35-4d35-9f66-43ac5032e31b None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:23.140460 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be326243-3d35-4d35-9f66-43ac5032e31b None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:23.142122 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 244/483] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:34:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:34:23.147475 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e2264d-e997-402b-a4b3-02d852095e64 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:23.147764 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e2264d-e997-402b-a4b3-02d852095e64 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:23.148045 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e2264d-e997-402b-a4b3-02d852095e64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:23.173093 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-f2e2264d-e997-402b-a4b3-02d852095e64 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:23.174225 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-f2e2264d-e997-402b-a4b3-02d852095e64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:23.204504 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be326243-3d35-4d35-9f66-43ac5032e31b None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:23.289847 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 240/484] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:34:23.368978 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:23.401431 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be326243-3d35-4d35-9f66-43ac5032e31b None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:23.457232 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768d3c74-b3bb-48d9-8aff-e923d35e2180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:23.457794 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768d3c74-b3bb-48d9-8aff-e923d35e2180 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:23.458075 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768d3c74-b3bb-48d9-8aff-e923d35e2180 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:23.458332 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768d3c74-b3bb-48d9-8aff-e923d35e2180 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:23.481791 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768d3c74-b3bb-48d9-8aff-e923d35e2180 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:34:23.487066 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-768d3c74-b3bb-48d9-8aff-e923d35e2180 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:34:23.487066 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:23.487066 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:23.487066 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:23.487066 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:23.487066 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:23.487066 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:23.487066 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:23.487066 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:23.487066 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:23.487066 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:23.487066 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:23.487066 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:34:23.487066 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:34:23.487066 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:34:23.487066 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:34:23.492191 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:34:23.492191 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:34:23.492191 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:34:23.492191 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:34:23.492191 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:23.492191 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:23.492191 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:23.492191 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:23.492191 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:23.492191 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:23.492191 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:23.492191 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:23.492191 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:23.492191 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:23.492191 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:23.492191 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:23.493034 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:23.493034 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:23.493034 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:23.493034 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:23.493034 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:34:23.493034 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:34:23.493034 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:34:23.493034 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:34:23.493034 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:34:23.493034 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:34:23.493034 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:34:23.493034 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:34:23.493034 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 245/485] 10.222.0.55 () {56 vars in 1102 bytes} [Wed Aug 24 08:34:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:23.494321 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e2264d-e997-402b-a4b3-02d852095e64 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:23.658988 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:23.665265 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e2264d-e997-402b-a4b3-02d852095e64 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:23.689483 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768d3c74-b3bb-48d9-8aff-e923d35e2180 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:23.724046 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919741ab-2890-4ff3-bf8a-b196cd05ef47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:23.725057 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919741ab-2890-4ff3-bf8a-b196cd05ef47 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:23.726864 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919741ab-2890-4ff3-bf8a-b196cd05ef47 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:23.727357 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919741ab-2890-4ff3-bf8a-b196cd05ef47 None admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:23.729226 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919741ab-2890-4ff3-bf8a-b196cd05ef47 None admin] RBAC: Adding query filter params (name=service) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:23.731544 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919741ab-2890-4ff3-bf8a-b196cd05ef47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:23.734587 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919741ab-2890-4ff3-bf8a-b196cd05ef47 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:23.751282 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 241/486] 10.222.0.55 () {56 vars in 1103 bytes} [Wed Aug 24 08:34:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:23.757717 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919741ab-2890-4ff3-bf8a-b196cd05ef47 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:23.928640 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768d3c74-b3bb-48d9-8aff-e923d35e2180 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:23.964772 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919741ab-2890-4ff3-bf8a-b196cd05ef47 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:24.004565 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204de03d-660a-46d2-ae6a-976ff2f67ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:24.005147 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204de03d-660a-46d2-ae6a-976ff2f67ef7 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:24.005460 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204de03d-660a-46d2-ae6a-976ff2f67ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:24.005749 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204de03d-660a-46d2-ae6a-976ff2f67ef7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:24.019400 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204de03d-660a-46d2-ae6a-976ff2f67ef7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:24.021023 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204de03d-660a-46d2-ae6a-976ff2f67ef7 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:24.021334 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa90cc23-1e68-490e-88e8-2dab21c4ce01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:24.023021 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa90cc23-1e68-490e-88e8-2dab21c4ce01 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:24.023785 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa90cc23-1e68-490e-88e8-2dab21c4ce01 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:24.024617 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa90cc23-1e68-490e-88e8-2dab21c4ce01 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:24.040586 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-204de03d-660a-46d2-ae6a-976ff2f67ef7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:34:24.040586 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:24.040586 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:24.040586 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:24.040586 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:24.040586 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:24.040586 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:24.040586 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:24.040586 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:24.040586 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:24.040586 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:24.040586 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:24.040586 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:34:24.040586 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:34:24.040586 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:34:24.040586 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:34:24.041558 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:24.041558 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:24.041558 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:24.041558 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:24.041558 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:24.041558 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:24.041558 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:24.041558 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:24.041558 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:24.041558 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:24.041558 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:24.041558 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:24.041558 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:24.041558 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:24.041558 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:24.041558 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:24.041558 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:34:24.042414 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:34:24.042414 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:34:24.042414 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:34:24.042414 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 242/487] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:34:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:24.046115 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204de03d-660a-46d2-ae6a-976ff2f67ef7 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:24.061813 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa90cc23-1e68-490e-88e8-2dab21c4ce01 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:34:24.067563 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-fa90cc23-1e68-490e-88e8-2dab21c4ce01 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:34:24.067563 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:24.067563 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:24.067563 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:24.067563 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:24.067563 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:24.067563 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:24.067563 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:24.067563 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:24.067563 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:24.067563 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:24.067563 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:24.067563 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:34:24.067563 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:34:24.067563 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:34:24.067563 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:34:24.068525 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:34:24.068525 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:34:24.068525 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:34:24.068525 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:34:24.068525 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:24.068525 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:24.068525 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:24.068525 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:24.068525 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:24.068525 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:24.068525 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:24.068525 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:24.068525 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:24.068525 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:24.068525 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:24.068525 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:24.069396 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:24.069396 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:24.069396 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:24.069396 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:24.069396 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:34:24.069396 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:34:24.069396 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:34:24.069396 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:34:24.069396 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:34:24.069396 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:34:24.069396 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:34:24.069396 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:34:24.070016 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 246/488] 10.222.0.55 () {56 vars in 1102 bytes} [Wed Aug 24 08:34:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 381 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:34:24.071033 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc640b6-179f-4c36-9a8f-ca4ac4302dbe None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:24.071318 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc640b6-179f-4c36-9a8f-ca4ac4302dbe None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:24.071596 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc640b6-179f-4c36-9a8f-ca4ac4302dbe None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:24.073012 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 247/489] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:34:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:34:24.077103 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:24.079893 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc640b6-179f-4c36-9a8f-ca4ac4302dbe None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:24.083533 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:24.225142 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204de03d-660a-46d2-ae6a-976ff2f67ef7 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:24.268422 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc640b6-179f-4c36-9a8f-ca4ac4302dbe None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:24.271590 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938a91a9-5017-4755-b5e7-3918d80102bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:24.273150 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938a91a9-5017-4755-b5e7-3918d80102bd None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:24.273314 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938a91a9-5017-4755-b5e7-3918d80102bd None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:24.273758 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938a91a9-5017-4755-b5e7-3918d80102bd None admin] PATH_INFO: `/v3/domains` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:24.274488 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938a91a9-5017-4755-b5e7-3918d80102bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:24.274798 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938a91a9-5017-4755-b5e7-3918d80102bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:24.277000 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938a91a9-5017-4755-b5e7-3918d80102bd None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:24.297778 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 243/490] 10.222.0.55 () {56 vars in 1109 bytes} [Wed Aug 24 08:34:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:34:24.299585 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4b2045-824c-4822-960b-c3df71753ec5 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:24.299868 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4b2045-824c-4822-960b-c3df71753ec5 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:24.300202 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4b2045-824c-4822-960b-c3df71753ec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:24.319318 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac56de0-a9dd-455f-8a7e-48711a2e2b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:24.320365 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac56de0-a9dd-455f-8a7e-48711a2e2b6c None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:24.321069 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac56de0-a9dd-455f-8a7e-48711a2e2b6c None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:24.321565 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac56de0-a9dd-455f-8a7e-48711a2e2b6c None admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:24.322495 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac56de0-a9dd-455f-8a7e-48711a2e2b6c None admin] RBAC: Adding query filter params (name=service) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:24.323738 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac56de0-a9dd-455f-8a7e-48711a2e2b6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:24.326180 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac56de0-a9dd-455f-8a7e-48711a2e2b6c None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:24.332722 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-ac4b2045-824c-4822-960b-c3df71753ec5 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:24.334916 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-ac4b2045-824c-4822-960b-c3df71753ec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:24.337796 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 248/491] 10.222.0.55 () {56 vars in 1103 bytes} [Wed Aug 24 08:34:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:24.339245 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac56de0-a9dd-455f-8a7e-48711a2e2b6c None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:24.469759 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:24.482413 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 244/492] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:34:24.482413 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4b2045-824c-4822-960b-c3df71753ec5 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:24.496362 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:24.677604 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac56de0-a9dd-455f-8a7e-48711a2e2b6c None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:24.737868 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4b2045-824c-4822-960b-c3df71753ec5 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:24.753380 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8583141a-810a-416b-b375-5e0c4bae2309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:24.754409 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8583141a-810a-416b-b375-5e0c4bae2309 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:24.754910 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8583141a-810a-416b-b375-5e0c4bae2309 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:24.755400 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8583141a-810a-416b-b375-5e0c4bae2309 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:24.770613 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8583141a-810a-416b-b375-5e0c4bae2309 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:24.773487 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8583141a-810a-416b-b375-5e0c4bae2309 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:24.802973 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-8583141a-810a-416b-b375-5e0c4bae2309 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 08:34:24.802973 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:24.802973 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:24.802973 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:24.802973 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:24.802973 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:24.802973 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:24.802973 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:24.802973 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:24.802973 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:24.802973 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:24.802973 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:24.802973 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:34:24.802973 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:34:24.802973 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:34:24.802973 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:34:24.804785 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:24.804785 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:24.804785 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:34:24.804785 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:34:24.804785 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:34:24.804785 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:34:24.804785 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:24.804785 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:24.804785 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:24.804785 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:24.804785 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:24.804785 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:24.804785 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:24.804785 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:24.804785 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:24.804785 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:24.804785 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:24.804785 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:24.807459 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:24.807459 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:24.807459 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:34:24.807459 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:34:24.807459 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:34:24.807459 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:34:24.807459 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:34:24.807459 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:34:24.807459 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 08:34:24.807459 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:34:24.807459 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 249/493] 10.222.0.55 () {56 vars in 1093 bytes} [Wed Aug 24 08:34:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 464 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:24.807459 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbc24c4-0508-4beb-bcb9-3a37eb859da0 None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:24.807459 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbc24c4-0508-4beb-bcb9-3a37eb859da0 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:24.808777 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbc24c4-0508-4beb-bcb9-3a37eb859da0 None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:24.809206 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 250/494] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:34:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:34:24.812636 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbc24c4-0508-4beb-bcb9-3a37eb859da0 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:24.816680 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6995f8f8-4b9f-4791-851c-1f7d5c1d4cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:24.818752 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6995f8f8-4b9f-4791-851c-1f7d5c1d4cb1 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:24.818752 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6995f8f8-4b9f-4791-851c-1f7d5c1d4cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:24.818752 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6995f8f8-4b9f-4791-851c-1f7d5c1d4cb1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:24.831925 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6995f8f8-4b9f-4791-851c-1f7d5c1d4cb1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:24.833435 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6995f8f8-4b9f-4791-851c-1f7d5c1d4cb1 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:24.850567 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-6995f8f8-4b9f-4791-851c-1f7d5c1d4cb1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:34:24.850567 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:24.850567 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:24.850567 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:24.850567 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:24.850567 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:24.850567 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:24.850567 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:24.850567 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:24.850567 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:24.850567 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:24.850567 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:24.850567 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:34:24.850567 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:34:24.850567 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:34:24.850567 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:34:24.852329 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:24.852329 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:24.852329 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:24.852329 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:24.852329 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:24.852329 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:24.852329 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:24.852329 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:24.852329 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:24.852329 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:24.852329 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:24.852329 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:24.852329 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:24.852329 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:24.852329 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:24.852329 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:24.852329 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:34:24.854488 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:34:24.854488 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:34:24.854488 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:34:24.857308 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 245/495] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:34:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 373 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:24.858951 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5761a345-80df-45ff-b9b4-c7838f453cd5 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:24.859246 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5761a345-80df-45ff-b9b4-c7838f453cd5 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:24.859521 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5761a345-80df-45ff-b9b4-c7838f453cd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:24.879704 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-5761a345-80df-45ff-b9b4-c7838f453cd5 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:24.881043 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-5761a345-80df-45ff-b9b4-c7838f453cd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:24.941477 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:24.973784 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 246/496] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:34:24.974800 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5761a345-80df-45ff-b9b4-c7838f453cd5 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:24.996262 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbc24c4-0508-4beb-bcb9-3a37eb859da0 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:25.058125 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:25.085036 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1120d80-8f1a-4d05-8574-760b0dce0041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:25.086563 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1120d80-8f1a-4d05-8574-760b0dce0041 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:25.087071 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1120d80-8f1a-4d05-8574-760b0dce0041 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:25.087564 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1120d80-8f1a-4d05-8574-760b0dce0041 None admin] PATH_INFO: `/v3/users` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:25.088987 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1120d80-8f1a-4d05-8574-760b0dce0041 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:25.089591 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1120d80-8f1a-4d05-8574-760b0dce0041 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:25.093006 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1120d80-8f1a-4d05-8574-760b0dce0041 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:25.119568 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5761a345-80df-45ff-b9b4-c7838f453cd5 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:25.168729 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 251/497] 10.222.0.55 () {56 vars in 1133 bytes} [Wed Aug 24 08:34:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:25.170411 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1120d80-8f1a-4d05-8574-760b0dce0041 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:25.188508 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb7a04a-c2e3-4430-bb55-c4963f729f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:25.189572 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb7a04a-c2e3-4430-bb55-c4963f729f1f None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:25.190048 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb7a04a-c2e3-4430-bb55-c4963f729f1f None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:25.190512 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb7a04a-c2e3-4430-bb55-c4963f729f1f None admin] PATH_INFO: `/v3/domains` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:25.191989 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb7a04a-c2e3-4430-bb55-c4963f729f1f None admin] RBAC: Adding query filter params (name=Default) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:25.195071 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb7a04a-c2e3-4430-bb55-c4963f729f1f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:25.195776 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb7a04a-c2e3-4430-bb55-c4963f729f1f None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:25.218423 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 247/498] 10.222.0.55 () {56 vars in 1109 bytes} [Wed Aug 24 08:34:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:25.220672 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a9805d-817d-487c-a396-977a7390e4ad None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:25.221183 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a9805d-817d-487c-a396-977a7390e4ad None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:25.221678 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a9805d-817d-487c-a396-977a7390e4ad None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:25.224260 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 248/499] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:34:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:34:25.233389 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a9805d-817d-487c-a396-977a7390e4ad None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:25.310083 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1120d80-8f1a-4d05-8574-760b0dce0041 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:25.314397 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:25.345657 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267c13dc-3f3b-4b8c-9701-b8783f368ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:25.346295 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267c13dc-3f3b-4b8c-9701-b8783f368ca0 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:25.346574 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267c13dc-3f3b-4b8c-9701-b8783f368ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:25.346842 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267c13dc-3f3b-4b8c-9701-b8783f368ca0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:25.359154 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267c13dc-3f3b-4b8c-9701-b8783f368ca0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:34:25.363937 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-267c13dc-3f3b-4b8c-9701-b8783f368ca0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:34:25.363937 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:25.363937 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:25.363937 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:25.363937 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:25.363937 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:25.363937 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:25.363937 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:25.363937 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:25.363937 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:25.363937 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:25.363937 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:25.363937 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:34:25.363937 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:34:25.363937 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:34:25.363937 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:34:25.365688 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:34:25.365688 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:34:25.365688 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:34:25.365688 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:34:25.365688 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:25.365688 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:25.365688 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:25.365688 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:25.365688 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:25.365688 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:25.365688 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:25.365688 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:25.365688 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:25.365688 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:25.365688 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:25.365688 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:25.367259 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:25.367259 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:25.367259 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:25.367259 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:25.367259 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:34:25.367259 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:34:25.367259 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:34:25.367259 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:34:25.367259 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:34:25.367259 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:34:25.367259 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:34:25.367259 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:34:25.367259 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 252/500] 10.222.0.55 () {56 vars in 1102 bytes} [Wed Aug 24 08:34:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:25.367259 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267c13dc-3f3b-4b8c-9701-b8783f368ca0 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:25.406038 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a9805d-817d-487c-a396-977a7390e4ad None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:25.422753 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:25.449581 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc27ff8-5646-456e-b0c4-73853b693031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:25.450321 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc27ff8-5646-456e-b0c4-73853b693031 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:25.450763 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc27ff8-5646-456e-b0c4-73853b693031 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:25.451204 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc27ff8-5646-456e-b0c4-73853b693031 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:25.500820 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267c13dc-3f3b-4b8c-9701-b8783f368ca0 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:25.547885 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a558a54c-ddac-4511-bcd8-b0244ebfee24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:25.548716 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a558a54c-ddac-4511-bcd8-b0244ebfee24 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:25.549111 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a558a54c-ddac-4511-bcd8-b0244ebfee24 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:25.549503 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a558a54c-ddac-4511-bcd8-b0244ebfee24 None admin] PATH_INFO: `/v3/users/glance` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:25.551417 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc27ff8-5646-456e-b0c4-73853b693031 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:25.553409 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc27ff8-5646-456e-b0c4-73853b693031 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:25.558772 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a558a54c-ddac-4511-bcd8-b0244ebfee24 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:25.560530 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a558a54c-ddac-4511-bcd8-b0244ebfee24 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:25.580246 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-a558a54c-ddac-4511-bcd8-b0244ebfee24 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 08:34:25.580246 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:25.580246 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:25.580246 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:25.580246 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:25.580246 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:25.580246 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:25.580246 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:25.580246 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:25.580246 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:25.580246 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:25.580246 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:25.580246 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:34:25.580246 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:34:25.580246 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:34:25.580246 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:34:25.581922 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:25.581922 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:25.581922 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:34:25.581922 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:34:25.581922 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:34:25.581922 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:34:25.581922 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:25.581922 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:25.581922 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:25.581922 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:25.581922 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:25.581922 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:25.581922 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:25.581922 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:25.581922 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:25.581922 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:25.581922 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:25.581922 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:25.584337 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:25.584337 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:25.584337 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:34:25.584337 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:34:25.584337 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:34:25.584337 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:34:25.584337 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:34:25.584337 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:34:25.584337 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 08:34:25.584337 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:34:25.584337 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 253/501] 10.222.0.55 () {56 vars in 1099 bytes} [Wed Aug 24 08:34:25 2022] GET /identity/v3/users/glance => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:25.584337 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9c50f4-c9a3-4093-89d8-cdfee25b9f27 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:25.586786 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9c50f4-c9a3-4093-89d8-cdfee25b9f27 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:25.586786 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9c50f4-c9a3-4093-89d8-cdfee25b9f27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:25.613250 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-2d9c50f4-c9a3-4093-89d8-cdfee25b9f27 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:25.615191 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-2d9c50f4-c9a3-4093-89d8-cdfee25b9f27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:25.624637 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:25.704864 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 249/502] 10.222.0.55 () {58 vars in 1282 bytes} [Wed Aug 24 08:34:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:34:25.706344 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc27ff8-5646-456e-b0c4-73853b693031 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:25.715303 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 254/503] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:34:25.717060 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9c50f4-c9a3-4093-89d8-cdfee25b9f27 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:25.758483 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:25.857952 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc27ff8-5646-456e-b0c4-73853b693031 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:25.890870 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9c50f4-c9a3-4093-89d8-cdfee25b9f27 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:25.900471 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b59696-c4f9-4670-a9c8-10000879c3d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:25.901145 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b59696-c4f9-4670-a9c8-10000879c3d0 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:25.901420 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b59696-c4f9-4670-a9c8-10000879c3d0 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:25.901706 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b59696-c4f9-4670-a9c8-10000879c3d0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:25.915470 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b59696-c4f9-4670-a9c8-10000879c3d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:34:25.920290 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-55b59696-c4f9-4670-a9c8-10000879c3d0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:34:25.920290 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:25.920290 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:25.920290 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:25.920290 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:25.920290 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:25.920290 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:25.920290 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:25.920290 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:25.920290 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:25.920290 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:25.920290 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:25.920290 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:34:25.920290 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:34:25.920290 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:34:25.920290 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:34:25.922386 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:34:25.922386 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:34:25.922386 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:34:25.922386 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:34:25.922386 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:25.922386 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:25.922386 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:25.922386 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:25.922386 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:25.922386 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:25.922386 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:25.922386 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:25.922386 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:25.922386 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:25.922386 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:25.922386 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:25.924025 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:25.924025 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:25.924025 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:25.924025 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:25.924025 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:34:25.924025 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:34:25.924025 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:34:25.924025 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:34:25.924025 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:34:25.924025 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:34:25.924025 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:34:25.924025 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:34:25.924025 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 250/504] 10.222.0.55 () {56 vars in 1102 bytes} [Wed Aug 24 08:34:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:25.924025 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b59696-c4f9-4670-a9c8-10000879c3d0 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:25.926248 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:25.966770 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6d814f-5cf9-45d6-84b0-1ea15551c2d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:25.967707 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6d814f-5cf9-45d6-84b0-1ea15551c2d9 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:25.968200 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6d814f-5cf9-45d6-84b0-1ea15551c2d9 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:25.968732 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6d814f-5cf9-45d6-84b0-1ea15551c2d9 None admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:25.969984 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6d814f-5cf9-45d6-84b0-1ea15551c2d9 None admin] RBAC: Adding query filter params (name=service) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:25.970530 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6d814f-5cf9-45d6-84b0-1ea15551c2d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:25.973246 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6d814f-5cf9-45d6-84b0-1ea15551c2d9 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:25.989173 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 255/505] 10.222.0.55 () {56 vars in 1103 bytes} [Wed Aug 24 08:34:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:25.990533 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6d814f-5cf9-45d6-84b0-1ea15551c2d9 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:26.038367 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b59696-c4f9-4670-a9c8-10000879c3d0 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:26.076608 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5217f8b3-ede6-4a8d-9848-c6930dc658d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:26.077221 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5217f8b3-ede6-4a8d-9848-c6930dc658d4 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:26.077494 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5217f8b3-ede6-4a8d-9848-c6930dc658d4 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:26.077761 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5217f8b3-ede6-4a8d-9848-c6930dc658d4 None admin] PATH_INFO: `/v3/users` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:26.078494 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5217f8b3-ede6-4a8d-9848-c6930dc658d4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:26.078791 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5217f8b3-ede6-4a8d-9848-c6930dc658d4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:26.080232 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5217f8b3-ede6-4a8d-9848-c6930dc658d4 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:26.118675 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 251/506] 10.222.0.55 () {56 vars in 1137 bytes} [Wed Aug 24 08:34:25 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:26.119428 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5217f8b3-ede6-4a8d-9848-c6930dc658d4 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:26.151776 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:26.160834 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6d814f-5cf9-45d6-84b0-1ea15551c2d9 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:26.198738 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:26.199518 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53861232-eec2-45fb-b2c0-d845dbc14e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:26.199965 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53861232-eec2-45fb-b2c0-d845dbc14e21 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:26.203342 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53861232-eec2-45fb-b2c0-d845dbc14e21 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:26.203342 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53861232-eec2-45fb-b2c0-d845dbc14e21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:26.214651 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53861232-eec2-45fb-b2c0-d845dbc14e21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:26.216303 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53861232-eec2-45fb-b2c0-d845dbc14e21 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:26.231094 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-53861232-eec2-45fb-b2c0-d845dbc14e21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:34:26.231094 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:26.231094 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:26.231094 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:26.231094 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:26.231094 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:26.231094 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:26.231094 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:26.231094 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:26.231094 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:26.231094 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:26.231094 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:26.231094 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:34:26.231094 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:34:26.231094 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:34:26.231094 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:34:26.232967 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:26.232967 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:26.232967 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:26.232967 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:26.232967 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:26.232967 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:26.232967 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:26.232967 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:26.232967 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:26.232967 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:26.232967 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:26.232967 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:26.232967 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:26.232967 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:26.232967 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:26.232967 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:26.232967 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:34:26.234877 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:34:26.234877 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:34:26.234877 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:34:26.234877 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 256/507] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:34:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:26.234877 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53861232-eec2-45fb-b2c0-d845dbc14e21 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:26.279446 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5217f8b3-ede6-4a8d-9848-c6930dc658d4 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:26.329931 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a422c148-0f3a-4f1a-bba8-f86117ab57f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:26.330870 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a422c148-0f3a-4f1a-bba8-f86117ab57f1 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:26.331311 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a422c148-0f3a-4f1a-bba8-f86117ab57f1 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:26.331746 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a422c148-0f3a-4f1a-bba8-f86117ab57f1 None admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:26.332870 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a422c148-0f3a-4f1a-bba8-f86117ab57f1 None admin] RBAC: Adding query filter params (name=service) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:26.333366 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a422c148-0f3a-4f1a-bba8-f86117ab57f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:26.335723 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a422c148-0f3a-4f1a-bba8-f86117ab57f1 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:26.354457 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 252/508] 10.222.0.55 () {56 vars in 1103 bytes} [Wed Aug 24 08:34:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:26.354457 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a422c148-0f3a-4f1a-bba8-f86117ab57f1 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:26.376614 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53861232-eec2-45fb-b2c0-d845dbc14e21 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:26.432342 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864c03c4-b967-4f6f-ad23-4a0aa3185003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:26.433819 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864c03c4-b967-4f6f-ad23-4a0aa3185003 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:26.434271 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864c03c4-b967-4f6f-ad23-4a0aa3185003 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:26.434714 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864c03c4-b967-4f6f-ad23-4a0aa3185003 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:26.453670 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864c03c4-b967-4f6f-ad23-4a0aa3185003 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:26.456357 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864c03c4-b967-4f6f-ad23-4a0aa3185003 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:26.467830 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:26.490698 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-864c03c4-b967-4f6f-ad23-4a0aa3185003 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:34:26.490698 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:26.490698 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:26.490698 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:26.490698 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:26.490698 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:26.490698 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:26.490698 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:26.490698 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:26.490698 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:26.490698 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:26.490698 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:26.490698 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:34:26.490698 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:34:26.490698 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:34:26.490698 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:34:26.492501 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:26.492501 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:26.492501 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:26.492501 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:26.492501 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:26.492501 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:26.492501 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:26.492501 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:26.492501 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:26.492501 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:26.492501 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:26.492501 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:26.492501 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:26.492501 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:26.492501 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:26.492501 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:26.492501 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:34:26.494198 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:34:26.494198 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:34:26.494198 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:34:26.494198 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 257/509] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:34:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:26.494198 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864c03c4-b967-4f6f-ad23-4a0aa3185003 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:26.518073 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a422c148-0f3a-4f1a-bba8-f86117ab57f1 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:26.546517 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:26.561707 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc5b8cc-cb72-434f-919b-4f0c4ba66e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:26.562648 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc5b8cc-cb72-434f-919b-4f0c4ba66e4b None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:26.563016 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc5b8cc-cb72-434f-919b-4f0c4ba66e4b None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:26.563457 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc5b8cc-cb72-434f-919b-4f0c4ba66e4b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:26.577022 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc5b8cc-cb72-434f-919b-4f0c4ba66e4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:26.578649 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc5b8cc-cb72-434f-919b-4f0c4ba66e4b None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:26.602099 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-dbc5b8cc-cb72-434f-919b-4f0c4ba66e4b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:34:26.602099 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:26.602099 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:26.602099 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:26.602099 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:26.602099 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:26.602099 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:26.602099 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:26.602099 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:26.602099 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:26.602099 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:26.602099 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:26.602099 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:34:26.602099 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:34:26.602099 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:34:26.602099 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:34:26.603458 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:26.603458 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:26.603458 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:26.603458 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:26.603458 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:26.603458 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:26.603458 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:26.603458 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:26.603458 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:26.603458 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:26.603458 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:26.603458 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:26.603458 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:26.603458 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:26.603458 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:26.603458 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:26.603458 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:34:26.604787 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:34:26.604787 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:34:26.604787 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:34:26.604787 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 253/510] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:34:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:26.605922 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc5b8cc-cb72-434f-919b-4f0c4ba66e4b None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:26.622371 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864c03c4-b967-4f6f-ad23-4a0aa3185003 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:26.666380 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a7ddc9-a00f-4c46-a485-cf9c3fbe92fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:26.667075 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a7ddc9-a00f-4c46-a485-cf9c3fbe92fa None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:26.667424 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a7ddc9-a00f-4c46-a485-cf9c3fbe92fa None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:26.668282 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a7ddc9-a00f-4c46-a485-cf9c3fbe92fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:26.737786 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc5b8cc-cb72-434f-919b-4f0c4ba66e4b None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:26.753418 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:26.757563 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a7ddc9-a00f-4c46-a485-cf9c3fbe92fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:26.759211 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a7ddc9-a00f-4c46-a485-cf9c3fbe92fa None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:26.781221 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a346d5c-691f-4757-b79d-b66f8f6633c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:26.781926 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a346d5c-691f-4757-b79d-b66f8f6633c5 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:26.782328 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a346d5c-691f-4757-b79d-b66f8f6633c5 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:26.782759 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a346d5c-691f-4757-b79d-b66f8f6633c5 None admin] PATH_INFO: `/v3/domains` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:26.783529 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a346d5c-691f-4757-b79d-b66f8f6633c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:26.783960 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a346d5c-691f-4757-b79d-b66f8f6633c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:26.785867 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a346d5c-691f-4757-b79d-b66f8f6633c5 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:26.799766 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 254/511] 10.222.0.55 () {56 vars in 1109 bytes} [Wed Aug 24 08:34:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:26.801224 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a346d5c-691f-4757-b79d-b66f8f6633c5 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:26.846640 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:26.885008 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 258/512] 10.222.0.55 () {58 vars in 1282 bytes} [Wed Aug 24 08:34:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:34:26.885866 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a7ddc9-a00f-4c46-a485-cf9c3fbe92fa None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:26.937750 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a346d5c-691f-4757-b79d-b66f8f6633c5 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:26.975692 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c92f6d5-2c83-4b7b-b840-49c60e3b5722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:26.976325 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c92f6d5-2c83-4b7b-b840-49c60e3b5722 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:26.976636 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c92f6d5-2c83-4b7b-b840-49c60e3b5722 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:26.976912 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c92f6d5-2c83-4b7b-b840-49c60e3b5722 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:26.987907 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c92f6d5-2c83-4b7b-b840-49c60e3b5722 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:26.990488 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c92f6d5-2c83-4b7b-b840-49c60e3b5722 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:27.013446 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-5c92f6d5-2c83-4b7b-b840-49c60e3b5722 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:34:27.013446 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:27.013446 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:27.013446 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:27.013446 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:27.013446 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:27.013446 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:27.013446 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:27.013446 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:27.013446 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:27.013446 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:27.013446 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:27.013446 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:34:27.013446 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:34:27.013446 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:34:27.013446 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:34:27.015380 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:27.015380 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:27.015380 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:27.015380 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:27.015380 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:27.015380 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:27.015380 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:27.015380 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:27.015380 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:27.015380 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:27.015380 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:27.015380 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:27.015380 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:27.015380 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:27.015380 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:27.015380 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:27.015380 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:34:27.017103 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:34:27.017103 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:34:27.017103 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:34:27.017103 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 255/513] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:34:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:27.017103 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c92f6d5-2c83-4b7b-b840-49c60e3b5722 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:27.052382 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:27.057279 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a7ddc9-a00f-4c46-a485-cf9c3fbe92fa None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:27.105820 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec12be7-8759-4740-9c3c-f00235bce4f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:27.106502 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec12be7-8759-4740-9c3c-f00235bce4f2 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:27.106917 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec12be7-8759-4740-9c3c-f00235bce4f2 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:27.107254 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec12be7-8759-4740-9c3c-f00235bce4f2 None admin] PATH_INFO: `/v3/domains` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:27.108028 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec12be7-8759-4740-9c3c-f00235bce4f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:27.108512 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec12be7-8759-4740-9c3c-f00235bce4f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:27.110274 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec12be7-8759-4740-9c3c-f00235bce4f2 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:27.133155 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:27.135826 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 259/514] 10.222.0.55 () {56 vars in 1109 bytes} [Wed Aug 24 08:34:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:27.138593 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec12be7-8759-4740-9c3c-f00235bce4f2 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:27.152670 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c92f6d5-2c83-4b7b-b840-49c60e3b5722 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:27.206528 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d868d2f7-284d-401b-bf04-362ded845ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:27.207513 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d868d2f7-284d-401b-bf04-362ded845ea7 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:27.207947 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d868d2f7-284d-401b-bf04-362ded845ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:27.208395 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d868d2f7-284d-401b-bf04-362ded845ea7 None admin] PATH_INFO: `/v3/domains` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:27.209511 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d868d2f7-284d-401b-bf04-362ded845ea7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:27.210017 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d868d2f7-284d-401b-bf04-362ded845ea7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:27.212317 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d868d2f7-284d-401b-bf04-362ded845ea7 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:27.230973 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 256/515] 10.222.0.55 () {56 vars in 1109 bytes} [Wed Aug 24 08:34:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:27.232413 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d868d2f7-284d-401b-bf04-362ded845ea7 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:27.278966 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec12be7-8759-4740-9c3c-f00235bce4f2 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:27.328989 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ba752c-32ff-4e27-8d07-0406e278b3db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:27.329751 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ba752c-32ff-4e27-8d07-0406e278b3db None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:27.330092 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ba752c-32ff-4e27-8d07-0406e278b3db None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:27.330427 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ba752c-32ff-4e27-8d07-0406e278b3db None admin] PATH_INFO: `/v3/projects/service` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:27.338544 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ba752c-32ff-4e27-8d07-0406e278b3db None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:27.340905 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ba752c-32ff-4e27-8d07-0406e278b3db None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:27.354090 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-33ba752c-32ff-4e27-8d07-0406e278b3db None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:34:27.354090 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:27.354090 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:27.354090 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:27.354090 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:27.354090 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:27.354090 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:27.354090 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:27.354090 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:27.354090 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:27.354090 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:27.354090 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:27.354090 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:34:27.354090 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:34:27.354090 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:34:27.356070 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:34:27.356070 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:27.356070 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:27.356070 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:27.356070 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:27.356070 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:27.356070 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:27.356070 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:27.356070 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:27.356070 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:27.356070 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:27.356070 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:27.356070 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:27.356070 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:27.356070 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:27.356070 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:27.356070 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:27.358442 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:34:27.358442 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:34:27.358442 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:34:27.358442 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:34:27.358442 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:34:27.358442 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:34:27.358442 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:34:27.358442 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:34:27.360762 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 260/516] 10.222.0.55 () {56 vars in 1111 bytes} [Wed Aug 24 08:34:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:27.362585 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ba752c-32ff-4e27-8d07-0406e278b3db None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:27.364744 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d868d2f7-284d-401b-bf04-362ded845ea7 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:27.388955 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:27.412429 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdde7fb6-129f-473b-a587-efab287ca6e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:27.416660 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdde7fb6-129f-473b-a587-efab287ca6e2 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:27.416660 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdde7fb6-129f-473b-a587-efab287ca6e2 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:27.416660 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdde7fb6-129f-473b-a587-efab287ca6e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:27.430476 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdde7fb6-129f-473b-a587-efab287ca6e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:27.432874 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdde7fb6-129f-473b-a587-efab287ca6e2 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:27.442236 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:27.453541 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-cdde7fb6-129f-473b-a587-efab287ca6e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:34:27.453541 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:27.453541 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:27.453541 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:27.453541 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:27.453541 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:27.453541 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:27.453541 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:27.453541 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:27.453541 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:27.453541 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:27.453541 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:27.453541 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:34:27.453541 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:34:27.453541 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:34:27.453541 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:34:27.457338 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:27.457338 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:27.457338 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:27.457338 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:27.457338 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:27.457338 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:27.457338 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:27.457338 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:27.457338 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:27.457338 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:27.457338 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:27.457338 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:27.457338 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:27.457338 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:27.457338 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:27.457338 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:27.457338 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:34:27.458208 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:34:27.458208 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:34:27.458208 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:34:27.458208 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 257/517] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:34:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:27.458208 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdde7fb6-129f-473b-a587-efab287ca6e2 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:27.512234 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ba752c-32ff-4e27-8d07-0406e278b3db None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:27.548891 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1894d6a1-b573-4070-a8aa-75d934d4d819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:27.549407 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1894d6a1-b573-4070-a8aa-75d934d4d819 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:27.549738 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1894d6a1-b573-4070-a8aa-75d934d4d819 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:27.550035 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1894d6a1-b573-4070-a8aa-75d934d4d819 None admin] PATH_INFO: `/v3/domains` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:27.550690 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1894d6a1-b573-4070-a8aa-75d934d4d819 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:27.550982 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1894d6a1-b573-4070-a8aa-75d934d4d819 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:27.552432 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1894d6a1-b573-4070-a8aa-75d934d4d819 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:27.565369 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 261/518] 10.222.0.55 () {56 vars in 1109 bytes} [Wed Aug 24 08:34:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:27.566246 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1894d6a1-b573-4070-a8aa-75d934d4d819 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:27.582350 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdde7fb6-129f-473b-a587-efab287ca6e2 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:27.636053 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c79fbd-6d57-4c91-9ebc-231443479b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:27.637636 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c79fbd-6d57-4c91-9ebc-231443479b80 None admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:27.638049 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c79fbd-6d57-4c91-9ebc-231443479b80 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:27.638487 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c79fbd-6d57-4c91-9ebc-231443479b80 None admin] PATH_INFO: `/v3/users` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:27.639552 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c79fbd-6d57-4c91-9ebc-231443479b80 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:27.642195 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c79fbd-6d57-4c91-9ebc-231443479b80 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:27.682179 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:27.707590 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1894d6a1-b573-4070-a8aa-75d934d4d819 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:27.750011 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc55bf6-88e3-477e-81f6-8db7e29368d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:27.750799 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc55bf6-88e3-477e-81f6-8db7e29368d9 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:27.751116 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc55bf6-88e3-477e-81f6-8db7e29368d9 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:27.751463 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc55bf6-88e3-477e-81f6-8db7e29368d9 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:27.761005 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc55bf6-88e3-477e-81f6-8db7e29368d9 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:27.762722 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc55bf6-88e3-477e-81f6-8db7e29368d9 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:27.776402 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-5fc55bf6-88e3-477e-81f6-8db7e29368d9 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 08:34:27.776402 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:27.776402 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:27.776402 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:27.776402 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:27.776402 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:27.776402 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:27.776402 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:27.776402 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:27.776402 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:27.776402 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:27.776402 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:27.776402 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:34:27.776402 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:34:27.776402 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:34:27.776402 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:34:27.778699 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:27.778699 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:27.778699 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:34:27.778699 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:34:27.778699 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:34:27.778699 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:34:27.778699 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:27.778699 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:27.778699 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:27.778699 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:27.778699 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:27.778699 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:27.778699 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:27.778699 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:27.778699 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:27.778699 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:27.778699 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:27.778699 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:27.780650 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:27.780650 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:27.780650 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:34:27.780650 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:34:27.780650 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:34:27.780650 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:34:27.780650 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:34:27.780650 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:34:27.780650 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 08:34:27.780650 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:34:27.780650 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 262/519] 10.222.0.55 () {56 vars in 1102 bytes} [Wed Aug 24 08:34:27 2022] GET /identity/v3/users/neutron => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:27.780650 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc55bf6-88e3-477e-81f6-8db7e29368d9 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:27.790184 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 258/520] 10.222.0.55 () {60 vars in 1132 bytes} [Wed Aug 24 08:34:27 2022] POST /identity/v3/users => generated 247 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:34:27.791170 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c79fbd-6d57-4c91-9ebc-231443479b80 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:27.870720 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:27.928228 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c79fbd-6d57-4c91-9ebc-231443479b80 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:27.942482 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc55bf6-88e3-477e-81f6-8db7e29368d9 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:27.974036 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b763ff-a63f-49d8-b1c3-51b57d8b6021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:27.975247 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b763ff-a63f-49d8-b1c3-51b57d8b6021 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:27.975798 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b763ff-a63f-49d8-b1c3-51b57d8b6021 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:27.976317 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b763ff-a63f-49d8-b1c3-51b57d8b6021 None admin] PATH_INFO: `/v3/domains` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:27.977658 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b763ff-a63f-49d8-b1c3-51b57d8b6021 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:27.978169 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b763ff-a63f-49d8-b1c3-51b57d8b6021 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:27.982680 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b763ff-a63f-49d8-b1c3-51b57d8b6021 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:27.990015 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17f6782-bd8c-4c9c-a7b4-c299762dea45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:27.990635 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17f6782-bd8c-4c9c-a7b4-c299762dea45 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:27.990916 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17f6782-bd8c-4c9c-a7b4-c299762dea45 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:27.991185 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17f6782-bd8c-4c9c-a7b4-c299762dea45 None admin] PATH_INFO: `/v3/projects` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:27.991889 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17f6782-bd8c-4c9c-a7b4-c299762dea45 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:27.992189 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17f6782-bd8c-4c9c-a7b4-c299762dea45 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:27.993733 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17f6782-bd8c-4c9c-a7b4-c299762dea45 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:28.005104 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 259/521] 10.222.0.55 () {56 vars in 1109 bytes} [Wed Aug 24 08:34:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:28.006021 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b763ff-a63f-49d8-b1c3-51b57d8b6021 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:28.014360 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:28.024426 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 263/522] 10.222.0.55 () {56 vars in 1148 bytes} [Wed Aug 24 08:34:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:28.025569 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17f6782-bd8c-4c9c-a7b4-c299762dea45 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:28.175656 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17f6782-bd8c-4c9c-a7b4-c299762dea45 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:28.191740 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b763ff-a63f-49d8-b1c3-51b57d8b6021 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:28.227418 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:28.245934 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602115b0-51b9-4fb4-bc91-1a5e0548c938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:28.246896 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602115b0-51b9-4fb4-bc91-1a5e0548c938 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:28.247423 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602115b0-51b9-4fb4-bc91-1a5e0548c938 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:28.247833 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602115b0-51b9-4fb4-bc91-1a5e0548c938 None admin] PATH_INFO: `/v3/users` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:28.249867 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602115b0-51b9-4fb4-bc91-1a5e0548c938 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:28.249867 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602115b0-51b9-4fb4-bc91-1a5e0548c938 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:28.252937 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602115b0-51b9-4fb4-bc91-1a5e0548c938 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:28.269847 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d61b62-1d8b-4174-b699-bccfe730ff77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:28.272056 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d61b62-1d8b-4174-b699-bccfe730ff77 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:28.273608 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d61b62-1d8b-4174-b699-bccfe730ff77 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:28.274191 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d61b62-1d8b-4174-b699-bccfe730ff77 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:28.282795 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 264/523] 10.222.0.55 () {56 vars in 1139 bytes} [Wed Aug 24 08:34:28 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:28.282795 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602115b0-51b9-4fb4-bc91-1a5e0548c938 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:28.299302 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d61b62-1d8b-4174-b699-bccfe730ff77 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:28.302678 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d61b62-1d8b-4174-b699-bccfe730ff77 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:28.331296 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-88d61b62-1d8b-4174-b699-bccfe730ff77 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 08:34:28.331296 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:28.331296 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:28.331296 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:28.331296 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:28.331296 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:28.331296 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:28.331296 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:28.331296 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:28.331296 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:28.331296 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:28.331296 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:28.331296 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:34:28.331296 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:34:28.331296 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:34:28.331296 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:34:28.332490 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:28.332490 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:28.332490 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:34:28.332490 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:34:28.332490 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:34:28.332490 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:34:28.332490 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:28.332490 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:28.332490 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:28.332490 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:28.332490 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:28.332490 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:28.332490 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:28.332490 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:28.332490 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:28.332490 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:28.332490 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:28.332490 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:28.333463 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:28.333463 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:28.333463 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:34:28.333463 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:34:28.333463 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:34:28.333463 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:34:28.333463 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:34:28.333463 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:34:28.333463 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 08:34:28.333463 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:34:28.341056 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 260/524] 10.222.0.55 () {56 vars in 1099 bytes} [Wed Aug 24 08:34:28 2022] GET /identity/v3/users/cinder => generated 84 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:28.342588 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d61b62-1d8b-4174-b699-bccfe730ff77 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:28.392011 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:28.479690 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602115b0-51b9-4fb4-bc91-1a5e0548c938 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:28.495200 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d61b62-1d8b-4174-b699-bccfe730ff77 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:28.541712 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccaa8d8-c175-479d-b7dc-eaea2624f799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:28.542863 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccaa8d8-c175-479d-b7dc-eaea2624f799 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:28.543842 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccaa8d8-c175-479d-b7dc-eaea2624f799 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:28.544319 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccaa8d8-c175-479d-b7dc-eaea2624f799 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:28.557554 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccaa8d8-c175-479d-b7dc-eaea2624f799 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:28.560366 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccaa8d8-c175-479d-b7dc-eaea2624f799 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:28.561079 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae8a909-13e4-4321-aa8f-252ef94288ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:28.562024 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae8a909-13e4-4321-aa8f-252ef94288ca None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:28.562408 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae8a909-13e4-4321-aa8f-252ef94288ca None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:28.562696 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae8a909-13e4-4321-aa8f-252ef94288ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:28.564130 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae8a909-13e4-4321-aa8f-252ef94288ca None admin] RBAC: Adding query filter params (role.id=bab90e02eebc473e9b911c8660ed9b76, scope.project.id=420b18725f2342159646fb86438318fd, user.id=f17da357e555420799ee681bffbcc268) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:28.564443 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae8a909-13e4-4321-aa8f-252ef94288ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:28.567633 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae8a909-13e4-4321-aa8f-252ef94288ca None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:28.584581 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 261/525] 10.222.0.55 () {56 vars in 1374 bytes} [Wed Aug 24 08:34:28 2022] GET /identity/v3/role_assignments?user.id=f17da357e555420799ee681bffbcc268&scope.project.id=420b18725f2342159646fb86438318fd&role.id=bab90e02eebc473e9b911c8660ed9b76 => generated 259 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:28.584581 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae8a909-13e4-4321-aa8f-252ef94288ca None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:28.584856 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-7ccaa8d8-c175-479d-b7dc-eaea2624f799 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:34:28.584856 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:28.584856 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:28.584856 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:28.584856 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:28.584856 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:28.584856 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:28.584856 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:28.584856 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:28.584856 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:28.584856 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:28.584856 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:28.584856 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:34:28.584856 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:34:28.584856 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:34:28.585778 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:34:28.585778 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:28.585778 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:28.585778 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:28.585778 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:28.585778 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:28.585778 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:28.585778 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:28.585778 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:28.585778 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:28.585778 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:28.585778 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:28.585778 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:28.585778 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:28.585778 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:28.585778 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:28.585778 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:28.586617 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:34:28.586617 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:34:28.586617 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:34:28.586617 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:34:28.586617 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:34:28.586617 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:34:28.586617 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:34:28.586617 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:34:28.592754 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 265/526] 10.222.0.55 () {56 vars in 1111 bytes} [Wed Aug 24 08:34:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:28.594294 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccaa8d8-c175-479d-b7dc-eaea2624f799 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:28.679570 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:28.797848 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccaa8d8-c175-479d-b7dc-eaea2624f799 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:28.807092 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae8a909-13e4-4321-aa8f-252ef94288ca None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:28.820063 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:28.844871 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf1756b-efc2-4aa0-a202-eda29b71a81b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:28.845834 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf1756b-efc2-4aa0-a202-eda29b71a81b None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:28.846295 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf1756b-efc2-4aa0-a202-eda29b71a81b None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:28.846808 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf1756b-efc2-4aa0-a202-eda29b71a81b None admin] PATH_INFO: `/v3/users` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:28.847736 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf1756b-efc2-4aa0-a202-eda29b71a81b None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:28.848253 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf1756b-efc2-4aa0-a202-eda29b71a81b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:28.850301 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf1756b-efc2-4aa0-a202-eda29b71a81b None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:28.877248 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3b6041-5a5b-46c9-9fdb-5f8f5f8f2aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:28.878449 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3b6041-5a5b-46c9-9fdb-5f8f5f8f2aa5 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:28.878909 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3b6041-5a5b-46c9-9fdb-5f8f5f8f2aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:28.879373 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3b6041-5a5b-46c9-9fdb-5f8f5f8f2aa5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:28.880212 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 266/527] 10.222.0.55 () {56 vars in 1137 bytes} [Wed Aug 24 08:34:28 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:28.881773 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf1756b-efc2-4aa0-a202-eda29b71a81b None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:28.969499 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3b6041-5a5b-46c9-9fdb-5f8f5f8f2aa5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:28.971639 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3b6041-5a5b-46c9-9fdb-5f8f5f8f2aa5 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:29.021280 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:29.069803 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf1756b-efc2-4aa0-a202-eda29b71a81b None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:29.114570 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c910e159-bb75-46ee-bd24-f8deea0014e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:29.115594 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c910e159-bb75-46ee-bd24-f8deea0014e8 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:29.116117 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c910e159-bb75-46ee-bd24-f8deea0014e8 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:29.116585 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c910e159-bb75-46ee-bd24-f8deea0014e8 None admin] PATH_INFO: `/v3/projects` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:29.117542 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c910e159-bb75-46ee-bd24-f8deea0014e8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:29.118028 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c910e159-bb75-46ee-bd24-f8deea0014e8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:29.119693 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c910e159-bb75-46ee-bd24-f8deea0014e8 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:29.128661 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 262/528] 10.222.0.55 () {58 vars in 1282 bytes} [Wed Aug 24 08:34:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 550 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:34:29.129994 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3b6041-5a5b-46c9-9fdb-5f8f5f8f2aa5 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:29.135089 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 267/529] 10.222.0.55 () {56 vars in 1148 bytes} [Wed Aug 24 08:34:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:29.137825 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c910e159-bb75-46ee-bd24-f8deea0014e8 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:29.185171 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:29.290097 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c910e159-bb75-46ee-bd24-f8deea0014e8 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:29.315573 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3b6041-5a5b-46c9-9fdb-5f8f5f8f2aa5 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:29.343948 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116db596-a919-4c4c-abb0-5cfeede38003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:29.344748 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116db596-a919-4c4c-abb0-5cfeede38003 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:29.345096 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116db596-a919-4c4c-abb0-5cfeede38003 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:29.345439 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116db596-a919-4c4c-abb0-5cfeede38003 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:29.362073 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116db596-a919-4c4c-abb0-5cfeede38003 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:29.364200 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116db596-a919-4c4c-abb0-5cfeede38003 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:29.385638 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-116db596-a919-4c4c-abb0-5cfeede38003 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:34:29.385638 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:29.385638 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:29.385638 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:29.385638 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:29.385638 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:29.385638 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:29.385638 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:29.385638 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:29.385638 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:29.385638 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:29.385638 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:29.385638 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:34:29.385638 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:34:29.385638 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:34:29.385638 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:34:29.387758 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:29.387758 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:29.387758 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:29.387758 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:29.387758 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:29.387758 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:29.387758 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:29.387758 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:29.387758 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:29.387758 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:29.387758 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:29.387758 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:29.387758 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:29.387758 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:29.387758 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:29.387758 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:29.387758 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:34:29.389660 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:34:29.389660 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:34:29.389660 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:34:29.389660 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 268/530] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:34:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:29.390176 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116db596-a919-4c4c-abb0-5cfeede38003 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:29.399233 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367a5ab1-3af4-441b-9c4b-cde4fd53d796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:29.400747 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367a5ab1-3af4-441b-9c4b-cde4fd53d796 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:29.401556 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367a5ab1-3af4-441b-9c4b-cde4fd53d796 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:29.402832 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367a5ab1-3af4-441b-9c4b-cde4fd53d796 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:29.421880 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:29.517566 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:29.539359 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116db596-a919-4c4c-abb0-5cfeede38003 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:29.546154 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367a5ab1-3af4-441b-9c4b-cde4fd53d796 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:29.549970 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367a5ab1-3af4-441b-9c4b-cde4fd53d796 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:29.592391 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e03329-c6d5-49a3-99f4-798c154b0112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:29.593190 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e03329-c6d5-49a3-99f4-798c154b0112 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:29.593588 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e03329-c6d5-49a3-99f4-798c154b0112 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:29.594036 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e03329-c6d5-49a3-99f4-798c154b0112 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:29.595297 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e03329-c6d5-49a3-99f4-798c154b0112 None admin] RBAC: Adding query filter params (role.id=bab90e02eebc473e9b911c8660ed9b76, scope.project.id=420b18725f2342159646fb86438318fd, user.id=5512043ca9a7421b821741ca88d50b07) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:29.596235 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e03329-c6d5-49a3-99f4-798c154b0112 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:29.598173 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e03329-c6d5-49a3-99f4-798c154b0112 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:29.612579 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 269/531] 10.222.0.55 () {56 vars in 1374 bytes} [Wed Aug 24 08:34:29 2022] GET /identity/v3/role_assignments?user.id=5512043ca9a7421b821741ca88d50b07&scope.project.id=420b18725f2342159646fb86438318fd&role.id=bab90e02eebc473e9b911c8660ed9b76 => generated 259 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:29.612579 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e03329-c6d5-49a3-99f4-798c154b0112 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:29.810774 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e03329-c6d5-49a3-99f4-798c154b0112 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:29.833984 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 263/532] 10.222.0.55 () {58 vars in 1282 bytes} [Wed Aug 24 08:34:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 705 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:34:29.842979 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367a5ab1-3af4-441b-9c4b-cde4fd53d796 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:29.893352 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec31357e-b054-4caa-839f-f80f5be19a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:29.894121 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec31357e-b054-4caa-839f-f80f5be19a85 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:29.894504 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec31357e-b054-4caa-839f-f80f5be19a85 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:29.894879 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec31357e-b054-4caa-839f-f80f5be19a85 None admin] PATH_INFO: `/v3/domains` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:29.895648 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec31357e-b054-4caa-839f-f80f5be19a85 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:29.896055 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec31357e-b054-4caa-839f-f80f5be19a85 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:29.897910 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec31357e-b054-4caa-839f-f80f5be19a85 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:29.915797 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 270/533] 10.222.0.55 () {56 vars in 1109 bytes} [Wed Aug 24 08:34:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:29.927114 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec31357e-b054-4caa-839f-f80f5be19a85 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:29.937923 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:29.954342 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:30.101651 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367a5ab1-3af4-441b-9c4b-cde4fd53d796 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:30.111266 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec31357e-b054-4caa-839f-f80f5be19a85 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:30.155158 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a4489f-cde6-47b6-8923-614137b4b6bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:30.155709 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a4489f-cde6-47b6-8923-614137b4b6bf None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:30.155994 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a4489f-cde6-47b6-8923-614137b4b6bf None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:30.156370 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a4489f-cde6-47b6-8923-614137b4b6bf None admin] PATH_INFO: `/v3/projects/service` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:30.167104 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a4489f-cde6-47b6-8923-614137b4b6bf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:30.168665 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a4489f-cde6-47b6-8923-614137b4b6bf None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:30.182912 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-14a4489f-cde6-47b6-8923-614137b4b6bf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:34:30.182912 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:30.182912 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:30.182912 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:30.182912 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:30.182912 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:30.182912 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:30.182912 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:30.182912 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:30.182912 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:30.182912 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:30.182912 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:30.182912 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:34:30.182912 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:34:30.182912 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:34:30.185337 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:34:30.185337 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:30.185337 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:30.185337 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:30.185337 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:30.185337 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:30.185337 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:30.185337 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:30.185337 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:30.185337 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:30.185337 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:30.185337 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:30.185337 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:30.185337 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:30.185337 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:30.185337 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:30.185337 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:30.186946 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:34:30.186946 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:34:30.186946 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:34:30.186946 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:34:30.186946 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:34:30.186946 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:34:30.186946 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:34:30.186946 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:34:30.186946 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 271/534] 10.222.0.55 () {56 vars in 1111 bytes} [Wed Aug 24 08:34:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:34:30.197033 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a4489f-cde6-47b6-8923-614137b4b6bf None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:30.237262 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcb5c17-944d-4110-82a5-01c67b966820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:30.238838 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcb5c17-944d-4110-82a5-01c67b966820 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:30.239473 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcb5c17-944d-4110-82a5-01c67b966820 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:30.240102 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcb5c17-944d-4110-82a5-01c67b966820 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:30.279290 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcb5c17-944d-4110-82a5-01c67b966820 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:30.282357 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcb5c17-944d-4110-82a5-01c67b966820 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:30.316610 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-ffcb5c17-944d-4110-82a5-01c67b966820 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:34:30.316610 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:30.316610 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:30.316610 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:30.316610 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:30.316610 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:30.316610 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:30.316610 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:30.316610 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:30.316610 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:30.316610 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:30.316610 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:30.316610 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:34:30.316610 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:34:30.316610 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:34:30.316610 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:34:30.318580 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:30.318580 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:30.318580 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:30.318580 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:30.318580 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:30.318580 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:30.318580 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:30.318580 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:30.318580 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:30.318580 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:30.318580 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:30.318580 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:30.318580 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:30.318580 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:30.318580 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:30.318580 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:30.318580 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:34:30.320525 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:34:30.320525 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:34:30.320525 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:34:30.323405 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 264/535] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:34:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 481 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:34:30.333260 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcb5c17-944d-4110-82a5-01c67b966820 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:30.463161 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:30.532360 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a4489f-cde6-47b6-8923-614137b4b6bf None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:30.578765 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:30.598946 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcb5c17-944d-4110-82a5-01c67b966820 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:30.676054 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91adb6ee-ddc1-482c-a9ab-02d3691c36c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:30.677470 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91adb6ee-ddc1-482c-a9ab-02d3691c36c2 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:30.678751 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8413172-d95c-4cf2-811e-edeef312c19c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:30.679482 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91adb6ee-ddc1-482c-a9ab-02d3691c36c2 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:30.680889 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91adb6ee-ddc1-482c-a9ab-02d3691c36c2 None admin] PATH_INFO: `/v3/domains` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:30.681533 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8413172-d95c-4cf2-811e-edeef312c19c None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:30.682973 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91adb6ee-ddc1-482c-a9ab-02d3691c36c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:30.684292 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8413172-d95c-4cf2-811e-edeef312c19c None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:30.685081 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91adb6ee-ddc1-482c-a9ab-02d3691c36c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:30.685781 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8413172-d95c-4cf2-811e-edeef312c19c None admin] PATH_INFO: `/v3/projects` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:30.688623 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91adb6ee-ddc1-482c-a9ab-02d3691c36c2 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:30.690168 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8413172-d95c-4cf2-811e-edeef312c19c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:30.691004 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8413172-d95c-4cf2-811e-edeef312c19c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:30.696241 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8413172-d95c-4cf2-811e-edeef312c19c None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:30.722563 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 272/536] 10.222.0.55 () {56 vars in 1148 bytes} [Wed Aug 24 08:34:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 526 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:30.729096 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 265/537] 10.222.0.55 () {56 vars in 1109 bytes} [Wed Aug 24 08:34:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 398 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:30.734728 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8413172-d95c-4cf2-811e-edeef312c19c None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:30.738442 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91adb6ee-ddc1-482c-a9ab-02d3691c36c2 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:31.004269 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91adb6ee-ddc1-482c-a9ab-02d3691c36c2 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:31.024878 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8413172-d95c-4cf2-811e-edeef312c19c None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:31.042687 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d48d4d2-41d2-45ef-8c9a-c1a57e933f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:31.043893 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d48d4d2-41d2-45ef-8c9a-c1a57e933f5c None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:31.044174 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d48d4d2-41d2-45ef-8c9a-c1a57e933f5c None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:31.044447 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d48d4d2-41d2-45ef-8c9a-c1a57e933f5c None admin] PATH_INFO: `/v3/projects/service` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:31.052490 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d48d4d2-41d2-45ef-8c9a-c1a57e933f5c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:31.054111 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d48d4d2-41d2-45ef-8c9a-c1a57e933f5c None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:31.067797 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-4d48d4d2-41d2-45ef-8c9a-c1a57e933f5c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:34:31.067797 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:31.067797 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:31.067797 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:31.067797 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:31.067797 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:31.067797 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:31.067797 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:31.067797 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:31.067797 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:31.067797 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:31.067797 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:31.067797 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:34:31.067797 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:34:31.067797 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:34:31.069861 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:34:31.069861 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:31.069861 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:31.069861 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:31.069861 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:31.069861 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:31.069861 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:31.069861 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:31.069861 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:31.069861 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:31.069861 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:31.069861 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:31.069861 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:31.069861 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:31.069861 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:31.069861 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:31.069861 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:31.071821 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:34:31.071821 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:34:31.071821 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:34:31.071821 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:34:31.071821 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:34:31.071821 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:34:31.071821 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:34:31.071821 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:34:31.071821 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 266/538] 10.222.0.55 () {56 vars in 1111 bytes} [Wed Aug 24 08:34:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:34:31.071821 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8732f0-1bc3-4404-b4a5-36f6cc79ceeb None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:31.071821 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8732f0-1bc3-4404-b4a5-36f6cc79ceeb None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:31.073889 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8732f0-1bc3-4404-b4a5-36f6cc79ceeb None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:31.073889 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 267/539] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:34:31 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:34:31.078701 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13260f7a-aff7-4d6b-84ed-ceeb6cc09367 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:31.079061 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13260f7a-aff7-4d6b-84ed-ceeb6cc09367 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:31.079307 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13260f7a-aff7-4d6b-84ed-ceeb6cc09367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:31.098616 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-13260f7a-aff7-4d6b-84ed-ceeb6cc09367 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:31.099842 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-13260f7a-aff7-4d6b-84ed-ceeb6cc09367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:31.100284 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:31.107391 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6cd6ce-7afb-47b8-9db8-4622c3f99cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:31.108858 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6cd6ce-7afb-47b8-9db8-4622c3f99cce None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:31.110258 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:31.111558 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6cd6ce-7afb-47b8-9db8-4622c3f99cce None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:31.112846 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6cd6ce-7afb-47b8-9db8-4622c3f99cce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:31.113639 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6cd6ce-7afb-47b8-9db8-4622c3f99cce None admin] RBAC: Adding query filter params (role.id=bab90e02eebc473e9b911c8660ed9b76, scope.project.id=420b18725f2342159646fb86438318fd, user.id=28b58388d1664510b0b507323bca5ae8) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:31.113954 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6cd6ce-7afb-47b8-9db8-4622c3f99cce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:31.115446 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6cd6ce-7afb-47b8-9db8-4622c3f99cce None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:31.125897 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 273/540] 10.222.0.55 () {56 vars in 1374 bytes} [Wed Aug 24 08:34:30 2022] GET /identity/v3/role_assignments?user.id=28b58388d1664510b0b507323bca5ae8&scope.project.id=420b18725f2342159646fb86438318fd&role.id=bab90e02eebc473e9b911c8660ed9b76 => generated 259 bytes in 391 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:34:31.128172 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6cd6ce-7afb-47b8-9db8-4622c3f99cce None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:31.199392 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 268/541] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 08:34:31.333398 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6cd6ce-7afb-47b8-9db8-4622c3f99cce None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:31.405875 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26580da4-e759-4a99-83da-3a6409293b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:31.406909 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26580da4-e759-4a99-83da-3a6409293b2f None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:31.407382 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26580da4-e759-4a99-83da-3a6409293b2f None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:31.407843 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26580da4-e759-4a99-83da-3a6409293b2f None admin] PATH_INFO: `/v3/projects` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:31.409160 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26580da4-e759-4a99-83da-3a6409293b2f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:31.409714 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26580da4-e759-4a99-83da-3a6409293b2f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:31.412572 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26580da4-e759-4a99-83da-3a6409293b2f None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:31.438759 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 274/542] 10.222.0.55 () {56 vars in 1148 bytes} [Wed Aug 24 08:34:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:31.448082 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13260f7a-aff7-4d6b-84ed-ceeb6cc09367 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:31.519178 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26580da4-e759-4a99-83da-3a6409293b2f None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:31.660918 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13260f7a-aff7-4d6b-84ed-ceeb6cc09367 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:31.690653 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:31.729574 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdd6433-90fb-4dd3-b6b9-85886f8ded4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:31.731606 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdd6433-90fb-4dd3-b6b9-85886f8ded4a None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:31.732386 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdd6433-90fb-4dd3-b6b9-85886f8ded4a None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:31.733078 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdd6433-90fb-4dd3-b6b9-85886f8ded4a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:31.734398 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdd6433-90fb-4dd3-b6b9-85886f8ded4a None admin] RBAC: Adding query filter params (role.id=bab90e02eebc473e9b911c8660ed9b76, scope.project.id=420b18725f2342159646fb86438318fd, user.id=fbbb9ac667f04c35a63b2689df18efed) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:31.734862 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdd6433-90fb-4dd3-b6b9-85886f8ded4a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:31.736924 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdd6433-90fb-4dd3-b6b9-85886f8ded4a None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:31.748190 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 269/543] 10.222.0.55 () {56 vars in 1374 bytes} [Wed Aug 24 08:34:31 2022] GET /identity/v3/role_assignments?user.id=fbbb9ac667f04c35a63b2689df18efed&scope.project.id=420b18725f2342159646fb86438318fd&role.id=bab90e02eebc473e9b911c8660ed9b76 => generated 259 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:31.846610 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91c7a6f-f578-416d-a855-cd5313361315 None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:31.847591 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91c7a6f-f578-416d-a855-cd5313361315 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:31.848353 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91c7a6f-f578-416d-a855-cd5313361315 None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:31.848515 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26580da4-e759-4a99-83da-3a6409293b2f None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:31.851799 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 270/544] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:34:31 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:34:31.867366 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03a9b7c-473d-4d43-b2a9-b3b5d0e95816 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:31.868228 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03a9b7c-473d-4d43-b2a9-b3b5d0e95816 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:31.868862 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03a9b7c-473d-4d43-b2a9-b3b5d0e95816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:31.894406 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff658673-56c9-44d8-87fc-107b582f6dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:31.895057 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff658673-56c9-44d8-87fc-107b582f6dab None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:31.895364 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff658673-56c9-44d8-87fc-107b582f6dab None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:31.895607 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff658673-56c9-44d8-87fc-107b582f6dab None admin] PATH_INFO: `/v3/roles/service` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:31.903366 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:31.909057 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff658673-56c9-44d8-87fc-107b582f6dab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:34:31.919008 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-ff658673-56c9-44d8-87fc-107b582f6dab None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:34:31.919008 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:31.919008 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:31.919008 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:31.919008 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:31.919008 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:31.919008 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:31.919008 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:31.919008 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:31.919008 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:31.919008 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:31.919008 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:31.919008 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:34:31.919008 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:34:31.919008 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:34:31.919008 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:34:31.920760 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:34:31.920760 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:34:31.920760 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:34:31.920760 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:34:31.920760 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:31.920760 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:31.920760 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:31.920760 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:31.920760 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:31.920760 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:31.920760 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:31.920760 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:31.920760 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:31.920760 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:31.920760 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:31.920760 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:31.922360 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:31.922360 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:31.922360 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:31.922360 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:31.922360 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:34:31.922360 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:34:31.922360 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:34:31.922360 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:34:31.922360 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:34:31.922360 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:34:31.922360 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:34:31.922360 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:34:31.922360 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 275/545] 10.222.0.55 () {56 vars in 1102 bytes} [Wed Aug 24 08:34:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 397 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:31.925599 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff658673-56c9-44d8-87fc-107b582f6dab None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:31.946362 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-c03a9b7c-473d-4d43-b2a9-b3b5d0e95816 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:31.948391 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-c03a9b7c-473d-4d43-b2a9-b3b5d0e95816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:32.052289 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff658673-56c9-44d8-87fc-107b582f6dab None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:32.085157 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:32.097848 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207a9993-8825-4588-ada4-8ef3fb429247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:32.098499 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207a9993-8825-4588-ada4-8ef3fb429247 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:32.103798 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207a9993-8825-4588-ada4-8ef3fb429247 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:32.103798 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207a9993-8825-4588-ada4-8ef3fb429247 None admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:32.103798 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207a9993-8825-4588-ada4-8ef3fb429247 None admin] RBAC: Adding query filter params (name=service) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:32.103798 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207a9993-8825-4588-ada4-8ef3fb429247 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:32.103798 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207a9993-8825-4588-ada4-8ef3fb429247 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:32.113030 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 276/546] 10.222.0.55 () {56 vars in 1103 bytes} [Wed Aug 24 08:34:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:34:32.126807 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207a9993-8825-4588-ada4-8ef3fb429247 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:32.236683 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 271/547] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:34:32.416974 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207a9993-8825-4588-ada4-8ef3fb429247 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:32.481833 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94882e6d-9462-46ea-a795-490410f32849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:32.482757 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94882e6d-9462-46ea-a795-490410f32849 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:32.483200 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94882e6d-9462-46ea-a795-490410f32849 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:32.483644 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94882e6d-9462-46ea-a795-490410f32849 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:32.513560 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94882e6d-9462-46ea-a795-490410f32849 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:32.516146 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94882e6d-9462-46ea-a795-490410f32849 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:32.549691 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03a9b7c-473d-4d43-b2a9-b3b5d0e95816 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:32.553247 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-94882e6d-9462-46ea-a795-490410f32849 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:34:32.553247 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:32.553247 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:32.553247 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:32.553247 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:32.553247 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:32.553247 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:32.553247 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:32.553247 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:32.553247 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:32.553247 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:32.553247 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:32.553247 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:34:32.553247 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:34:32.553247 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:34:32.553247 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:34:32.556079 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:32.556079 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:32.556079 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:32.556079 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:32.556079 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:32.556079 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:32.556079 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:32.556079 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:32.556079 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:32.556079 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:32.556079 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:32.556079 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:32.556079 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:32.556079 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:32.556079 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:32.556079 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:32.556079 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:34:32.560791 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:34:32.560791 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:34:32.560791 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:34:32.560791 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 277/548] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:34:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 431 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:32.566329 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94882e6d-9462-46ea-a795-490410f32849 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:32.652471 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:32.671267 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03a9b7c-473d-4d43-b2a9-b3b5d0e95816 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:32.755182 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b377c42-8e4f-440c-afad-188856f10f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:32.756954 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b377c42-8e4f-440c-afad-188856f10f86 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:32.757723 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b377c42-8e4f-440c-afad-188856f10f86 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:32.758496 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b377c42-8e4f-440c-afad-188856f10f86 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:32.768045 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:32.788279 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b377c42-8e4f-440c-afad-188856f10f86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:34:32.790948 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94882e6d-9462-46ea-a795-490410f32849 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:32.800004 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-0b377c42-8e4f-440c-afad-188856f10f86 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:34:32.800004 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:32.800004 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:32.800004 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:32.800004 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:32.800004 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:32.800004 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:32.800004 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:32.800004 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:32.800004 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:32.800004 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:32.800004 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:32.800004 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:34:32.800004 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:34:32.800004 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:34:32.800004 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:34:32.801744 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:34:32.801744 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:34:32.801744 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:34:32.801744 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:34:32.801744 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:32.801744 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:32.801744 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:32.801744 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:32.801744 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:32.801744 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:32.801744 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:32.801744 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:32.801744 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:32.801744 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:32.801744 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:32.801744 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:32.803952 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:32.803952 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:32.803952 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:32.803952 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:32.803952 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:34:32.803952 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:34:32.803952 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:34:32.803952 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:34:32.803952 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:34:32.803952 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:34:32.803952 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:34:32.803952 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:34:32.810315 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 272/549] 10.222.0.55 () {56 vars in 1102 bytes} [Wed Aug 24 08:34:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:32.823939 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b377c42-8e4f-440c-afad-188856f10f86 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:32.844312 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d8dc4a-e51e-486a-bc5b-902b40842af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:32.844966 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d8dc4a-e51e-486a-bc5b-902b40842af6 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:32.845245 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d8dc4a-e51e-486a-bc5b-902b40842af6 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:32.845521 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d8dc4a-e51e-486a-bc5b-902b40842af6 None admin] PATH_INFO: `/v3/domains` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:32.846185 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d8dc4a-e51e-486a-bc5b-902b40842af6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:32.846489 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d8dc4a-e51e-486a-bc5b-902b40842af6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:32.854135 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d8dc4a-e51e-486a-bc5b-902b40842af6 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:32.863979 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 278/550] 10.222.0.55 () {56 vars in 1109 bytes} [Wed Aug 24 08:34:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:32.874528 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d8dc4a-e51e-486a-bc5b-902b40842af6 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:33.066715 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b377c42-8e4f-440c-afad-188856f10f86 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:33.090665 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d8dc4a-e51e-486a-bc5b-902b40842af6 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:33.095369 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:33.138173 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4e414f-d40f-407d-9de2-967b78db700d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:33.139020 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4e414f-d40f-407d-9de2-967b78db700d None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:33.139406 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4e414f-d40f-407d-9de2-967b78db700d None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:33.139785 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4e414f-d40f-407d-9de2-967b78db700d None admin] PATH_INFO: `/v3/users/swift` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:33.147139 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07934d11-9a53-491f-bf2f-4d42aabdc286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:33.148710 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07934d11-9a53-491f-bf2f-4d42aabdc286 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:33.149239 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4e414f-d40f-407d-9de2-967b78db700d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:33.149699 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07934d11-9a53-491f-bf2f-4d42aabdc286 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:33.150369 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07934d11-9a53-491f-bf2f-4d42aabdc286 None admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:33.151053 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4e414f-d40f-407d-9de2-967b78db700d None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:33.152015 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07934d11-9a53-491f-bf2f-4d42aabdc286 None admin] RBAC: Adding query filter params (name=service) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:33.153307 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07934d11-9a53-491f-bf2f-4d42aabdc286 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:33.156116 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07934d11-9a53-491f-bf2f-4d42aabdc286 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:33.166062 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-ee4e414f-d40f-407d-9de2-967b78db700d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 08:34:33.166062 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:33.166062 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:33.166062 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:33.166062 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:33.166062 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:33.166062 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:33.166062 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:33.166062 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:33.166062 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:33.166062 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:33.166062 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:33.166062 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:34:33.166062 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:34:33.166062 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:34:33.166062 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:34:33.168697 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:33.168697 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:33.168697 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:34:33.168697 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:34:33.168697 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:34:33.168697 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:34:33.168697 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:33.168697 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:33.168697 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:33.168697 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:33.168697 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:33.168697 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:33.168697 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:33.168697 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:33.168697 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:33.168697 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:33.168697 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:33.168697 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:33.172954 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:33.172954 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:33.172954 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:34:33.172954 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:34:33.172954 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:34:33.172954 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:34:33.172954 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:34:33.172954 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:34:33.172954 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 08:34:33.172954 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:34:33.172954 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 279/551] 10.222.0.55 () {56 vars in 1096 bytes} [Wed Aug 24 08:34:32 2022] GET /identity/v3/users/swift => generated 83 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:34:33.172954 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52367718-0a72-4aa3-9331-3e67f5e69cbb None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:33.172954 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52367718-0a72-4aa3-9331-3e67f5e69cbb None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:33.182904 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52367718-0a72-4aa3-9331-3e67f5e69cbb None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:33.182904 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 280/552] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:34:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:34:33.183341 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 273/553] 10.222.0.55 () {56 vars in 1103 bytes} [Wed Aug 24 08:34:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:34:33.187297 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52367718-0a72-4aa3-9331-3e67f5e69cbb None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:33.191242 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07934d11-9a53-491f-bf2f-4d42aabdc286 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:33.338674 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:33.339604 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52367718-0a72-4aa3-9331-3e67f5e69cbb None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:33.384986 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b964f17-008f-40d7-87f4-202e477756c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:33.385637 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b964f17-008f-40d7-87f4-202e477756c8 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:33.385922 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b964f17-008f-40d7-87f4-202e477756c8 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:33.386186 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b964f17-008f-40d7-87f4-202e477756c8 None admin] PATH_INFO: `/v3/users` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:33.387468 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b964f17-008f-40d7-87f4-202e477756c8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:33.387775 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b964f17-008f-40d7-87f4-202e477756c8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:33.389299 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b964f17-008f-40d7-87f4-202e477756c8 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:33.420585 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 281/554] 10.222.0.55 () {56 vars in 1135 bytes} [Wed Aug 24 08:34:33 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:33.420585 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357e47e7-4960-4d14-bd35-bbce0fb220d2 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:33.420585 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357e47e7-4960-4d14-bd35-bbce0fb220d2 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:33.420585 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357e47e7-4960-4d14-bd35-bbce0fb220d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:33.432514 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:33.440504 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-357e47e7-4960-4d14-bd35-bbce0fb220d2 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:33.442217 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-357e47e7-4960-4d14-bd35-bbce0fb220d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:33.471608 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07934d11-9a53-491f-bf2f-4d42aabdc286 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:33.540574 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 282/555] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:34:33.540574 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357e47e7-4960-4d14-bd35-bbce0fb220d2 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:33.554201 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20b8c9d-6eb8-44aa-a8d5-de5310b9803e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:33.554868 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20b8c9d-6eb8-44aa-a8d5-de5310b9803e None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:33.556581 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20b8c9d-6eb8-44aa-a8d5-de5310b9803e None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:33.556883 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20b8c9d-6eb8-44aa-a8d5-de5310b9803e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:33.580899 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20b8c9d-6eb8-44aa-a8d5-de5310b9803e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:33.582405 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20b8c9d-6eb8-44aa-a8d5-de5310b9803e None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:33.609597 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-c20b8c9d-6eb8-44aa-a8d5-de5310b9803e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:34:33.609597 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:33.609597 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:33.609597 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:33.609597 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:33.609597 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:33.609597 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:33.609597 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:33.609597 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:33.609597 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:33.609597 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:33.609597 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:33.609597 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:34:33.609597 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:34:33.609597 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:34:33.609597 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:34:33.611365 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:33.611365 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:33.611365 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:33.611365 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:33.611365 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:33.611365 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:33.611365 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:33.611365 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:33.611365 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:33.611365 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:33.611365 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:33.611365 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:33.611365 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:33.611365 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:33.611365 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:33.611365 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:33.611365 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:34:33.613241 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:34:33.613241 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:34:33.613241 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:34:33.619024 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 274/556] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:34:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 427 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:33.619981 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20b8c9d-6eb8-44aa-a8d5-de5310b9803e None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:33.825131 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357e47e7-4960-4d14-bd35-bbce0fb220d2 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:33.849689 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20b8c9d-6eb8-44aa-a8d5-de5310b9803e None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:33.869974 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:33.924569 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fa342e-e6db-4a66-97e1-e37461b2477d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:33.926049 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fa342e-e6db-4a66-97e1-e37461b2477d None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:33.926802 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fa342e-e6db-4a66-97e1-e37461b2477d None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:33.927480 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fa342e-e6db-4a66-97e1-e37461b2477d None admin] PATH_INFO: `/v3/domains` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:33.928964 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fa342e-e6db-4a66-97e1-e37461b2477d None admin] RBAC: Adding query filter params (name=Default) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:33.929738 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fa342e-e6db-4a66-97e1-e37461b2477d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:33.932261 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be858cb-c88d-4e6e-8c5c-d3e16225a16c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:33.932928 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be858cb-c88d-4e6e-8c5c-d3e16225a16c None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:33.933212 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be858cb-c88d-4e6e-8c5c-d3e16225a16c None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:33.933487 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be858cb-c88d-4e6e-8c5c-d3e16225a16c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:33.934420 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fa342e-e6db-4a66-97e1-e37461b2477d None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:33.947369 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:33.963723 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 275/557] 10.222.0.55 () {56 vars in 1109 bytes} [Wed Aug 24 08:34:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:33.970184 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fa342e-e6db-4a66-97e1-e37461b2477d None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:34.047680 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be858cb-c88d-4e6e-8c5c-d3e16225a16c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:34.050082 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be858cb-c88d-4e6e-8c5c-d3e16225a16c None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:34.176657 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 283/558] 10.222.0.55 () {58 vars in 1282 bytes} [Wed Aug 24 08:34:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 636 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:34:34.176657 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be858cb-c88d-4e6e-8c5c-d3e16225a16c None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:34.213352 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:34.233823 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fa342e-e6db-4a66-97e1-e37461b2477d None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:34.294445 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e59d2e-2aee-4fdf-a0fe-f07c971e3487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:34.295699 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e59d2e-2aee-4fdf-a0fe-f07c971e3487 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:34.296464 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e59d2e-2aee-4fdf-a0fe-f07c971e3487 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:34.297091 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e59d2e-2aee-4fdf-a0fe-f07c971e3487 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:34.300086 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be858cb-c88d-4e6e-8c5c-d3e16225a16c None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:34.310324 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e59d2e-2aee-4fdf-a0fe-f07c971e3487 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:34.312220 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e59d2e-2aee-4fdf-a0fe-f07c971e3487 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:34.336202 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-72e59d2e-2aee-4fdf-a0fe-f07c971e3487 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 08:34:34.336202 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:34.336202 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:34.336202 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:34.336202 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:34.336202 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:34.336202 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:34.336202 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:34.336202 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:34.336202 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:34.336202 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:34.336202 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:34.336202 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:34:34.336202 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:34:34.336202 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:34:34.336202 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:34:34.338400 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:34.338400 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:34.338400 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:34:34.338400 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:34:34.338400 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:34:34.338400 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:34:34.338400 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:34.338400 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:34.338400 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:34.338400 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:34.338400 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:34.338400 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:34.338400 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:34.338400 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:34.338400 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:34.338400 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:34.338400 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:34.338400 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:34.342133 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:34.342133 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:34.342133 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:34:34.342133 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:34:34.342133 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:34:34.342133 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:34:34.342133 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:34:34.342133 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:34:34.342133 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 08:34:34.342133 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:34:34.342133 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 276/559] 10.222.0.55 () {56 vars in 1093 bytes} [Wed Aug 24 08:34:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 369 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:34:34.342133 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf44f3c-fde3-4102-8235-bf3f1285238d None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:34.342133 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf44f3c-fde3-4102-8235-bf3f1285238d None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:34.345608 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf44f3c-fde3-4102-8235-bf3f1285238d None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:34.345608 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 277/560] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:34:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:34:34.346070 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf44f3c-fde3-4102-8235-bf3f1285238d None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:34.372318 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4e24ef-d962-4daa-b2d1-1116f3fd4e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:34.373009 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4e24ef-d962-4daa-b2d1-1116f3fd4e91 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:34.373290 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4e24ef-d962-4daa-b2d1-1116f3fd4e91 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:34.373564 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4e24ef-d962-4daa-b2d1-1116f3fd4e91 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:34.389141 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4e24ef-d962-4daa-b2d1-1116f3fd4e91 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:34:34.394332 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:34.395860 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-9e4e24ef-d962-4daa-b2d1-1116f3fd4e91 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:34:34.395860 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:34.395860 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:34.395860 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:34.395860 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:34.395860 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:34.395860 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:34.395860 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:34.395860 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:34.395860 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:34.395860 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:34.395860 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:34.395860 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:34:34.395860 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:34:34.395860 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:34:34.395860 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:34:34.398002 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:34:34.398002 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:34:34.398002 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:34:34.398002 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:34:34.398002 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:34.398002 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:34.398002 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:34.398002 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:34.398002 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:34.398002 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:34.398002 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:34.398002 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:34.398002 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:34.398002 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:34.398002 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:34.398002 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:34.398847 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:34.398847 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:34.398847 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:34.398847 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:34.398847 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:34:34.398847 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:34:34.398847 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:34:34.398847 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:34:34.398847 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:34:34.398847 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:34:34.398847 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:34:34.398847 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:34:34.401935 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 284/561] 10.222.0.55 () {56 vars in 1102 bytes} [Wed Aug 24 08:34:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:34.403400 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4e24ef-d962-4daa-b2d1-1116f3fd4e91 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:34.530645 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:34.538131 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4e24ef-d962-4daa-b2d1-1116f3fd4e91 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:34.560583 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf44f3c-fde3-4102-8235-bf3f1285238d None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:34.587560 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5536cd6d-d6e9-4d1d-9e1b-33432c2b62e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:34.588613 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5536cd6d-d6e9-4d1d-9e1b-33432c2b62e9 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:34.589001 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5536cd6d-d6e9-4d1d-9e1b-33432c2b62e9 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:34.589399 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5536cd6d-d6e9-4d1d-9e1b-33432c2b62e9 None admin] PATH_INFO: `/v3/users` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:34.590918 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5536cd6d-d6e9-4d1d-9e1b-33432c2b62e9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:34.591345 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5536cd6d-d6e9-4d1d-9e1b-33432c2b62e9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:34.593299 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5536cd6d-d6e9-4d1d-9e1b-33432c2b62e9 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:34.623917 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 285/562] 10.222.0.55 () {56 vars in 1133 bytes} [Wed Aug 24 08:34:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:34.626735 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b963a9-f227-46a2-8d23-cd8babf5aeb8 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:34.627231 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b963a9-f227-46a2-8d23-cd8babf5aeb8 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:34.627716 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b963a9-f227-46a2-8d23-cd8babf5aeb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:34.632903 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a81cc4-a7df-41f7-8d48-ade218756deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:34.633939 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a81cc4-a7df-41f7-8d48-ade218756deb None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:34.634293 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a81cc4-a7df-41f7-8d48-ade218756deb None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:34.634661 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a81cc4-a7df-41f7-8d48-ade218756deb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:34.656277 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a81cc4-a7df-41f7-8d48-ade218756deb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:34.660695 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a81cc4-a7df-41f7-8d48-ade218756deb None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:34.663290 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-15b963a9-f227-46a2-8d23-cd8babf5aeb8 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:34.664711 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-15b963a9-f227-46a2-8d23-cd8babf5aeb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:34.680523 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-d0a81cc4-a7df-41f7-8d48-ade218756deb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:34:34.680523 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:34.680523 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:34.680523 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:34.680523 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:34.680523 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:34.680523 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:34.680523 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:34.680523 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:34.680523 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:34.680523 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:34.680523 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:34.680523 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:34:34.680523 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:34:34.680523 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:34:34.680523 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:34:34.690134 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:34.690134 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:34.690134 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:34.690134 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:34.690134 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:34.690134 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:34.690134 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:34.690134 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:34.690134 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:34.690134 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:34.690134 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:34.690134 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:34.690134 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:34.690134 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:34.690134 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:34.690134 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:34.690134 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:34:34.691806 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:34:34.691806 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:34:34.691806 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:34:34.691806 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 278/563] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:34:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:34.691806 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a81cc4-a7df-41f7-8d48-ade218756deb None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:34.766527 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 286/564] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:34:34.778123 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b963a9-f227-46a2-8d23-cd8babf5aeb8 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:34.822546 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:34.856804 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a81cc4-a7df-41f7-8d48-ade218756deb None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:34.919194 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09835eb1-5e7b-4aae-9af1-df8215d9d11a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:34.921110 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09835eb1-5e7b-4aae-9af1-df8215d9d11a None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:34.921110 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09835eb1-5e7b-4aae-9af1-df8215d9d11a None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:34.921942 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09835eb1-5e7b-4aae-9af1-df8215d9d11a None admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:34.922881 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09835eb1-5e7b-4aae-9af1-df8215d9d11a None admin] RBAC: Adding query filter params (name=service) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:34.923515 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09835eb1-5e7b-4aae-9af1-df8215d9d11a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:34.926068 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09835eb1-5e7b-4aae-9af1-df8215d9d11a None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:34.942101 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 279/565] 10.222.0.55 () {56 vars in 1103 bytes} [Wed Aug 24 08:34:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:34.943684 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09835eb1-5e7b-4aae-9af1-df8215d9d11a None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:34.950340 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:34.981360 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b963a9-f227-46a2-8d23-cd8babf5aeb8 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:35.046657 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20345054-5d4d-49f7-b5ac-56f4d05a4777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:35.048055 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20345054-5d4d-49f7-b5ac-56f4d05a4777 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:35.048843 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20345054-5d4d-49f7-b5ac-56f4d05a4777 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:35.049574 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20345054-5d4d-49f7-b5ac-56f4d05a4777 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:35.072414 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20345054-5d4d-49f7-b5ac-56f4d05a4777 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:35.079939 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20345054-5d4d-49f7-b5ac-56f4d05a4777 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:35.108611 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-20345054-5d4d-49f7-b5ac-56f4d05a4777 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:34:35.108611 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:35.108611 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:35.108611 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:35.108611 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:35.108611 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:35.108611 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:35.108611 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:35.108611 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:35.108611 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:35.108611 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:35.108611 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:35.108611 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:34:35.108611 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:34:35.108611 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:34:35.108611 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:34:35.110831 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:35.110831 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:35.110831 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:35.110831 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:35.110831 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:35.110831 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:35.110831 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:35.110831 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:35.110831 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:35.110831 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:35.110831 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:35.110831 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:35.110831 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:35.110831 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:35.110831 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:35.110831 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:35.110831 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:34:35.113069 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:34:35.113069 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:34:35.113069 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:34:35.115677 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 287/566] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:34:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:35.123000 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20345054-5d4d-49f7-b5ac-56f4d05a4777 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:35.138141 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09835eb1-5e7b-4aae-9af1-df8215d9d11a None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:35.217154 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c4d451-79a7-4d85-ba95-8a575f2a738e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:35.218089 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c4d451-79a7-4d85-ba95-8a575f2a738e None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:35.218551 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c4d451-79a7-4d85-ba95-8a575f2a738e None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:35.219266 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c4d451-79a7-4d85-ba95-8a575f2a738e None admin] PATH_INFO: `/v3/domains` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:35.220101 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c4d451-79a7-4d85-ba95-8a575f2a738e None admin] RBAC: Adding query filter params (name=Default) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:35.220600 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c4d451-79a7-4d85-ba95-8a575f2a738e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:35.222547 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c4d451-79a7-4d85-ba95-8a575f2a738e None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:35.239469 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 280/567] 10.222.0.55 () {56 vars in 1109 bytes} [Wed Aug 24 08:34:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:35.240502 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c4d451-79a7-4d85-ba95-8a575f2a738e None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:35.244876 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:35.301581 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20345054-5d4d-49f7-b5ac-56f4d05a4777 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:35.349974 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a4270e-7724-4cec-a802-cacc31c3a679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:35.350819 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a4270e-7724-4cec-a802-cacc31c3a679 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:35.351221 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a4270e-7724-4cec-a802-cacc31c3a679 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:35.351628 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a4270e-7724-4cec-a802-cacc31c3a679 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:35.356815 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:35.372268 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a4270e-7724-4cec-a802-cacc31c3a679 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:35.373758 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c4d451-79a7-4d85-ba95-8a575f2a738e None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:35.374950 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a4270e-7724-4cec-a802-cacc31c3a679 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:35.397514 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-68a4270e-7724-4cec-a802-cacc31c3a679 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:34:35.397514 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:35.397514 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:35.397514 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:35.397514 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:35.397514 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:35.397514 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:35.397514 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:35.397514 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:35.397514 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:35.397514 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:35.397514 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:35.397514 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:34:35.397514 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:34:35.397514 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:34:35.397514 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:34:35.399355 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:35.399355 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:35.399355 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:35.399355 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:35.399355 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:35.399355 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:35.399355 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:35.399355 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:35.399355 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:35.399355 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:35.399355 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:35.399355 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:35.399355 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:35.399355 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:35.399355 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:35.399355 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:35.399355 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:34:35.401135 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:34:35.401135 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:34:35.401135 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:34:35.401135 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 288/568] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:34:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:35.401135 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a4270e-7724-4cec-a802-cacc31c3a679 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:35.422208 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ceed39-0cfa-452a-9984-ab68741f67cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:35.422969 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ceed39-0cfa-452a-9984-ab68741f67cb None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:35.423355 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ceed39-0cfa-452a-9984-ab68741f67cb None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:35.423728 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ceed39-0cfa-452a-9984-ab68741f67cb None admin] PATH_INFO: `/v3/domains` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:35.424855 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ceed39-0cfa-452a-9984-ab68741f67cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:35.425350 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ceed39-0cfa-452a-9984-ab68741f67cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:35.428120 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ceed39-0cfa-452a-9984-ab68741f67cb None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:35.455263 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 281/569] 10.222.0.55 () {56 vars in 1109 bytes} [Wed Aug 24 08:34:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:35.455263 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7eca46-1898-45dc-a654-a2020225faae None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:35.455263 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7eca46-1898-45dc-a654-a2020225faae None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:35.455263 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7eca46-1898-45dc-a654-a2020225faae None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:35.455263 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 282/570] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:34:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:34:35.456847 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7eca46-1898-45dc-a654-a2020225faae None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:35.560440 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:35.566788 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a4270e-7724-4cec-a802-cacc31c3a679 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:35.611219 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7eca46-1898-45dc-a654-a2020225faae None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:35.615377 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3273d5-cb26-40a1-b258-168545bde50a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:35.616469 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3273d5-cb26-40a1-b258-168545bde50a None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:35.617010 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3273d5-cb26-40a1-b258-168545bde50a None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:35.617493 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3273d5-cb26-40a1-b258-168545bde50a None admin] PATH_INFO: `/v3/roles/service` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:35.637941 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3273d5-cb26-40a1-b258-168545bde50a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:34:35.648365 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-3a3273d5-cb26-40a1-b258-168545bde50a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:34:35.648365 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:35.648365 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:35.648365 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:35.648365 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:35.648365 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:35.648365 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:35.648365 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:35.648365 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:35.648365 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:35.648365 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:35.648365 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:35.648365 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:34:35.648365 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:34:35.648365 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:34:35.648365 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:34:35.650217 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:34:35.650217 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:34:35.650217 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:34:35.650217 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:34:35.650217 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:35.650217 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:35.650217 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:35.650217 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:35.650217 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:35.650217 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:35.650217 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:35.650217 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:35.650217 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:35.650217 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:35.650217 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:35.650217 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:35.651447 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:35.651447 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:35.651447 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:35.651447 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:35.651447 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:34:35.651447 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:34:35.651447 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:34:35.651447 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:34:35.651447 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:34:35.651447 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:34:35.651447 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:34:35.651447 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:34:35.651447 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 289/571] 10.222.0.55 () {56 vars in 1102 bytes} [Wed Aug 24 08:34:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:35.652818 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3273d5-cb26-40a1-b258-168545bde50a None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:35.655409 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5218b2-2123-44c2-8b95-a3ab18adcc6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:35.656200 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5218b2-2123-44c2-8b95-a3ab18adcc6a None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:35.656613 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5218b2-2123-44c2-8b95-a3ab18adcc6a None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:35.657003 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5218b2-2123-44c2-8b95-a3ab18adcc6a None admin] PATH_INFO: `/v3/projects/service` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:35.665547 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5218b2-2123-44c2-8b95-a3ab18adcc6a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:35.667019 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5218b2-2123-44c2-8b95-a3ab18adcc6a None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:35.679495 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-bc5218b2-2123-44c2-8b95-a3ab18adcc6a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:34:35.679495 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:35.679495 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:35.679495 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:35.679495 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:35.679495 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:35.679495 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:35.679495 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:35.679495 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:35.679495 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:35.679495 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:35.679495 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:35.679495 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:34:35.679495 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:34:35.679495 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:34:35.681418 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:34:35.681418 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:35.681418 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:35.681418 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:35.681418 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:35.681418 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:35.681418 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:35.681418 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:35.681418 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:35.681418 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:35.681418 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:35.681418 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:35.681418 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:35.681418 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:35.681418 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:35.681418 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:35.681418 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:35.684357 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:34:35.684357 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:34:35.684357 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:34:35.684357 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:34:35.684357 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:34:35.684357 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:34:35.684357 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:34:35.684357 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:34:35.684357 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 283/572] 10.222.0.55 () {56 vars in 1111 bytes} [Wed Aug 24 08:34:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:35.684357 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5218b2-2123-44c2-8b95-a3ab18adcc6a None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:35.730349 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:35.798816 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3273d5-cb26-40a1-b258-168545bde50a None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:35.812664 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5218b2-2123-44c2-8b95-a3ab18adcc6a None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:35.840212 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820b360d-c6ff-4497-a965-435740a460a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:35.840872 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820b360d-c6ff-4497-a965-435740a460a8 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:35.841159 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820b360d-c6ff-4497-a965-435740a460a8 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:35.841432 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820b360d-c6ff-4497-a965-435740a460a8 None admin] PATH_INFO: `/v3/domains` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:35.842084 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820b360d-c6ff-4497-a965-435740a460a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:35.842389 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820b360d-c6ff-4497-a965-435740a460a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:35.843503 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:35.843811 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820b360d-c6ff-4497-a965-435740a460a8 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:35.857395 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 290/573] 10.222.0.55 () {56 vars in 1109 bytes} [Wed Aug 24 08:34:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:35.858683 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab7c9ca-af34-4317-8910-c931430a3985 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:35.858984 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab7c9ca-af34-4317-8910-c931430a3985 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:35.859248 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab7c9ca-af34-4317-8910-c931430a3985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:35.863767 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e65c5e-de3d-48e3-81af-f828cd75f5de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:35.864813 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e65c5e-de3d-48e3-81af-f828cd75f5de None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:35.865305 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e65c5e-de3d-48e3-81af-f828cd75f5de None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:35.865803 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e65c5e-de3d-48e3-81af-f828cd75f5de None admin] PATH_INFO: `/v3/projects/service` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:35.879155 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-dab7c9ca-af34-4317-8910-c931430a3985 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:35.879434 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e65c5e-de3d-48e3-81af-f828cd75f5de None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:35.880391 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-dab7c9ca-af34-4317-8910-c931430a3985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:35.882413 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e65c5e-de3d-48e3-81af-f828cd75f5de None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:35.903508 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-f3e65c5e-de3d-48e3-81af-f828cd75f5de None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:34:35.903508 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:35.903508 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:35.903508 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:35.903508 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:35.903508 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:35.903508 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:35.903508 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:35.903508 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:35.903508 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:35.903508 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:35.903508 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:35.903508 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:34:35.903508 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:34:35.903508 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:34:35.904763 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:34:35.904763 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:35.904763 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:35.904763 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:35.904763 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:35.904763 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:35.904763 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:35.904763 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:35.904763 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:35.904763 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:35.904763 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:35.904763 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:35.904763 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:35.904763 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:35.904763 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:35.904763 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:35.904763 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:35.905843 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:34:35.905843 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:34:35.905843 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:34:35.905843 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:34:35.905843 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:34:35.905843 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:34:35.905843 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:34:35.905843 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:34:35.905843 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 284/574] 10.222.0.55 () {56 vars in 1111 bytes} [Wed Aug 24 08:34:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:35.907104 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e65c5e-de3d-48e3-81af-f828cd75f5de None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:35.965978 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 291/575] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:34:35.966850 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab7c9ca-af34-4317-8910-c931430a3985 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:36.014121 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:36.069890 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e65c5e-de3d-48e3-81af-f828cd75f5de None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:36.104228 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab7c9ca-af34-4317-8910-c931430a3985 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:36.123113 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789759a1-a8fe-41f0-be6c-448fe292f355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:36.123646 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789759a1-a8fe-41f0-be6c-448fe292f355 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:36.123931 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789759a1-a8fe-41f0-be6c-448fe292f355 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:36.124194 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789759a1-a8fe-41f0-be6c-448fe292f355 None admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:36.125053 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789759a1-a8fe-41f0-be6c-448fe292f355 None admin] RBAC: Adding query filter params (name=service) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:36.125354 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789759a1-a8fe-41f0-be6c-448fe292f355 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:36.126788 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789759a1-a8fe-41f0-be6c-448fe292f355 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:36.145284 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 285/576] 10.222.0.55 () {56 vars in 1103 bytes} [Wed Aug 24 08:34:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:36.145284 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789759a1-a8fe-41f0-be6c-448fe292f355 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:36.179756 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136fdc66-b137-4ed1-b22a-a48af11f1a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:36.180804 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136fdc66-b137-4ed1-b22a-a48af11f1a4b None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:36.181225 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136fdc66-b137-4ed1-b22a-a48af11f1a4b None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:36.181660 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136fdc66-b137-4ed1-b22a-a48af11f1a4b None admin] PATH_INFO: `/v3/projects` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:36.182836 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136fdc66-b137-4ed1-b22a-a48af11f1a4b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:36.183333 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136fdc66-b137-4ed1-b22a-a48af11f1a4b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:36.185874 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136fdc66-b137-4ed1-b22a-a48af11f1a4b None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:36.213303 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 292/577] 10.222.0.55 () {56 vars in 1148 bytes} [Wed Aug 24 08:34:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:36.214705 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136fdc66-b137-4ed1-b22a-a48af11f1a4b None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:36.235956 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:36.300011 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789759a1-a8fe-41f0-be6c-448fe292f355 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:36.336735 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56cadce-3618-4952-afd4-562f94b91377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:36.337368 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56cadce-3618-4952-afd4-562f94b91377 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:36.337647 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56cadce-3618-4952-afd4-562f94b91377 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:36.337929 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56cadce-3618-4952-afd4-562f94b91377 None admin] PATH_INFO: `/v3/users/glance` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:36.346554 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56cadce-3618-4952-afd4-562f94b91377 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:36.348001 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56cadce-3618-4952-afd4-562f94b91377 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:36.359947 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-a56cadce-3618-4952-afd4-562f94b91377 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 08:34:36.359947 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:36.359947 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:36.359947 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:36.359947 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:36.359947 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:36.359947 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:36.359947 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:36.359947 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:36.359947 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:36.359947 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:36.359947 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:36.359947 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:34:36.359947 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:34:36.359947 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:34:36.359947 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:34:36.361884 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:36.361884 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:36.361884 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:34:36.361884 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:34:36.361884 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:34:36.361884 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:34:36.361884 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:36.361884 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:36.361884 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:36.361884 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:36.361884 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:36.361884 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:36.361884 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:36.361884 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:36.361884 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:36.361884 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:36.361884 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:36.361884 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:36.363050 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:36.363050 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:36.363050 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:34:36.363050 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:34:36.363050 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:34:36.363050 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:34:36.363050 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:34:36.363050 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:34:36.363050 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 08:34:36.363050 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:34:36.363050 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 286/578] 10.222.0.55 () {56 vars in 1099 bytes} [Wed Aug 24 08:34:36 2022] GET /identity/v3/users/glance => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:36.363050 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56cadce-3618-4952-afd4-562f94b91377 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:36.378653 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:36.383951 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136fdc66-b137-4ed1-b22a-a48af11f1a4b None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:36.428089 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5604992a-4323-4ae6-a79b-3105c6864333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:36.429177 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5604992a-4323-4ae6-a79b-3105c6864333 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:36.429696 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5604992a-4323-4ae6-a79b-3105c6864333 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:36.430151 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5604992a-4323-4ae6-a79b-3105c6864333 None admin] PATH_INFO: `/v3/projects` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:36.431292 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5604992a-4323-4ae6-a79b-3105c6864333 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:36.431810 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5604992a-4323-4ae6-a79b-3105c6864333 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:36.434349 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5604992a-4323-4ae6-a79b-3105c6864333 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:36.455414 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 293/579] 10.222.0.55 () {56 vars in 1148 bytes} [Wed Aug 24 08:34:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:36.457057 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5604992a-4323-4ae6-a79b-3105c6864333 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:36.474757 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56cadce-3618-4952-afd4-562f94b91377 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:36.477524 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:36.511000 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4635a790-087c-46c5-8d03-525c845f0206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:36.511731 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4635a790-087c-46c5-8d03-525c845f0206 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:36.512651 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4635a790-087c-46c5-8d03-525c845f0206 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:36.512988 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4635a790-087c-46c5-8d03-525c845f0206 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:36.525624 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4635a790-087c-46c5-8d03-525c845f0206 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:36.527172 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4635a790-087c-46c5-8d03-525c845f0206 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:36.542982 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-4635a790-087c-46c5-8d03-525c845f0206 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:34:36.542982 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:36.542982 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:36.542982 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:36.542982 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:36.542982 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:36.542982 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:36.542982 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:36.542982 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:36.542982 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:36.542982 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:36.542982 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:36.542982 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:34:36.542982 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:34:36.542982 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:34:36.542982 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:34:36.544908 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:36.544908 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:36.544908 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:36.544908 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:36.544908 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:36.544908 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:36.544908 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:36.544908 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:36.544908 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:36.544908 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:36.544908 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:36.544908 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:36.544908 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:36.544908 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:36.544908 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:36.544908 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:36.544908 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:34:36.546651 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:34:36.546651 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:34:36.546651 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:34:36.546651 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 287/580] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:34:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:36.546651 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4635a790-087c-46c5-8d03-525c845f0206 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:36.615261 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5604992a-4323-4ae6-a79b-3105c6864333 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:36.666632 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af9fa3d-e0e1-4558-b451-59a03271af20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:36.667264 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af9fa3d-e0e1-4558-b451-59a03271af20 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:36.667713 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af9fa3d-e0e1-4558-b451-59a03271af20 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:36.668053 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af9fa3d-e0e1-4558-b451-59a03271af20 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:36.669045 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af9fa3d-e0e1-4558-b451-59a03271af20 None admin] RBAC: Adding query filter params (role.id=bab90e02eebc473e9b911c8660ed9b76, scope.project.id=420b18725f2342159646fb86438318fd, user.id=a0ebb5ccabef45378f0c1da69721e051) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:36.669500 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af9fa3d-e0e1-4558-b451-59a03271af20 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:36.671010 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af9fa3d-e0e1-4558-b451-59a03271af20 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:36.677279 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 294/581] 10.222.0.55 () {56 vars in 1374 bytes} [Wed Aug 24 08:34:36 2022] GET /identity/v3/role_assignments?user.id=a0ebb5ccabef45378f0c1da69721e051&scope.project.id=420b18725f2342159646fb86438318fd&role.id=bab90e02eebc473e9b911c8660ed9b76 => generated 259 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:36.678125 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af9fa3d-e0e1-4558-b451-59a03271af20 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:36.679064 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4635a790-087c-46c5-8d03-525c845f0206 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:36.721441 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:36.743172 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b60acf-f0d9-4ea6-875b-0decf434e822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:36.744157 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b60acf-f0d9-4ea6-875b-0decf434e822 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:36.744492 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b60acf-f0d9-4ea6-875b-0decf434e822 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:36.744859 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b60acf-f0d9-4ea6-875b-0decf434e822 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:36.761255 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b60acf-f0d9-4ea6-875b-0decf434e822 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:34:36.772812 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-03b60acf-f0d9-4ea6-875b-0decf434e822 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:34:36.772812 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:36.772812 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:36.772812 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:36.772812 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:36.772812 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:36.772812 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:36.772812 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:36.772812 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:36.772812 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:36.772812 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:36.772812 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:36.772812 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:34:36.772812 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:34:36.772812 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:34:36.772812 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:34:36.773980 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:34:36.773980 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:34:36.773980 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:34:36.773980 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:34:36.773980 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:36.773980 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:36.773980 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:36.773980 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:36.773980 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:36.773980 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:36.773980 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:36.773980 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:36.773980 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:36.773980 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:36.773980 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:36.773980 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:36.774935 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:36.774935 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:36.774935 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:36.774935 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:36.774935 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:34:36.774935 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:34:36.774935 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:34:36.774935 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:34:36.774935 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:34:36.774935 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:34:36.774935 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:34:36.774935 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:34:36.781017 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 288/582] 10.222.0.55 () {56 vars in 1102 bytes} [Wed Aug 24 08:34:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:36.782307 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b60acf-f0d9-4ea6-875b-0decf434e822 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:36.813199 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:36.836616 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af9fa3d-e0e1-4558-b451-59a03271af20 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:36.887399 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498e33f1-8db1-4579-92d3-e1686180a5ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:36.888030 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498e33f1-8db1-4579-92d3-e1686180a5ae None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:36.888311 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498e33f1-8db1-4579-92d3-e1686180a5ae None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:36.888691 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498e33f1-8db1-4579-92d3-e1686180a5ae None admin] PATH_INFO: `/v3/users` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:36.889436 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498e33f1-8db1-4579-92d3-e1686180a5ae None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:36.889746 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498e33f1-8db1-4579-92d3-e1686180a5ae None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:36.891214 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498e33f1-8db1-4579-92d3-e1686180a5ae None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:36.910748 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 295/583] 10.222.0.55 () {56 vars in 1137 bytes} [Wed Aug 24 08:34:36 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:36.911604 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498e33f1-8db1-4579-92d3-e1686180a5ae None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:36.930375 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b60acf-f0d9-4ea6-875b-0decf434e822 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:36.972181 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983d014d-3298-4572-9f1f-a221196e8ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:36.973030 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983d014d-3298-4572-9f1f-a221196e8ca0 None admin] REQUEST_METHOD: `PUT` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:36.973309 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983d014d-3298-4572-9f1f-a221196e8ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:36.973581 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983d014d-3298-4572-9f1f-a221196e8ca0 None admin] PATH_INFO: `/v3/projects/420b18725f2342159646fb86438318fd/users/f17da357e555420799ee681bffbcc268/roles/bab90e02eebc473e9b911c8660ed9b76` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:36.990542 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983d014d-3298-4572-9f1f-a221196e8ca0 None admin] RBAC: Authorizing `identity:create_grant(project_id=420b18725f2342159646fb86438318fd, user_id=f17da357e555420799ee681bffbcc268, role_id=bab90e02eebc473e9b911c8660ed9b76)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:36.992260 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983d014d-3298-4572-9f1f-a221196e8ca0 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:37.033348 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:37.054397 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498e33f1-8db1-4579-92d3-e1686180a5ae None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:37.091426 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d380fb4-3036-4a5d-a48a-97d7845f9106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:37.092374 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d380fb4-3036-4a5d-a48a-97d7845f9106 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:37.092677 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d380fb4-3036-4a5d-a48a-97d7845f9106 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:37.092952 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d380fb4-3036-4a5d-a48a-97d7845f9106 None admin] PATH_INFO: `/v3/domains` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:37.093624 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d380fb4-3036-4a5d-a48a-97d7845f9106 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:37.093925 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d380fb4-3036-4a5d-a48a-97d7845f9106 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:37.095423 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d380fb4-3036-4a5d-a48a-97d7845f9106 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:37.104588 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 289/584] 10.222.0.55 () {58 vars in 1439 bytes} [Wed Aug 24 08:34:36 2022] PUT /identity/v3/projects/420b18725f2342159646fb86438318fd/users/f17da357e555420799ee681bffbcc268/roles/bab90e02eebc473e9b911c8660ed9b76 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:34:37.104588 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983d014d-3298-4572-9f1f-a221196e8ca0 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:37.117920 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 296/585] 10.222.0.55 () {56 vars in 1109 bytes} [Wed Aug 24 08:34:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:37.118826 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d380fb4-3036-4a5d-a48a-97d7845f9106 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:37.242417 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:37.311464 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983d014d-3298-4572-9f1f-a221196e8ca0 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:37.351567 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d380fb4-3036-4a5d-a48a-97d7845f9106 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:37.382892 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3661de-2d83-44e3-844a-1df5dc12b99a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:37.384480 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3661de-2d83-44e3-844a-1df5dc12b99a None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:37.384982 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3661de-2d83-44e3-844a-1df5dc12b99a None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:37.385441 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3661de-2d83-44e3-844a-1df5dc12b99a None admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:37.386977 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3661de-2d83-44e3-844a-1df5dc12b99a None admin] RBAC: Adding query filter params (name=service) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:37.387502 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3661de-2d83-44e3-844a-1df5dc12b99a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:37.390261 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3661de-2d83-44e3-844a-1df5dc12b99a None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:37.408689 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 290/586] 10.222.0.55 () {56 vars in 1103 bytes} [Wed Aug 24 08:34:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:37.412827 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3661de-2d83-44e3-844a-1df5dc12b99a None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:37.430876 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0901d9d-1a8c-4829-b0b6-5339128aac63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:37.431900 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0901d9d-1a8c-4829-b0b6-5339128aac63 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:37.432364 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0901d9d-1a8c-4829-b0b6-5339128aac63 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:37.432843 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0901d9d-1a8c-4829-b0b6-5339128aac63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:37.452656 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:37.457455 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0901d9d-1a8c-4829-b0b6-5339128aac63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:37.460396 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0901d9d-1a8c-4829-b0b6-5339128aac63 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:37.492798 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-d0901d9d-1a8c-4829-b0b6-5339128aac63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:34:37.492798 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:37.492798 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:37.492798 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:37.492798 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:37.492798 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:37.492798 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:37.492798 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:37.492798 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:37.492798 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:37.492798 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:37.492798 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:37.492798 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:34:37.492798 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:34:37.492798 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:34:37.492798 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:34:37.494635 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:37.494635 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:37.494635 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:37.494635 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:37.494635 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:37.494635 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:37.494635 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:37.494635 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:37.494635 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:37.494635 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:37.494635 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:37.494635 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:37.494635 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:37.494635 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:37.494635 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:37.494635 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:37.494635 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:34:37.497119 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:34:37.497119 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:34:37.497119 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:34:37.497119 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 297/587] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:34:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 377 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:37.497954 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0901d9d-1a8c-4829-b0b6-5339128aac63 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:37.626247 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3661de-2d83-44e3-844a-1df5dc12b99a None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:37.684425 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:37.689433 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0901d9d-1a8c-4829-b0b6-5339128aac63 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:37.699835 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891f4f5d-3d91-499a-b86b-5761cafb8663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:37.700877 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891f4f5d-3d91-499a-b86b-5761cafb8663 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:37.701349 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891f4f5d-3d91-499a-b86b-5761cafb8663 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:37.701809 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891f4f5d-3d91-499a-b86b-5761cafb8663 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:37.715965 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891f4f5d-3d91-499a-b86b-5761cafb8663 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:37.719197 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891f4f5d-3d91-499a-b86b-5761cafb8663 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:37.741172 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-891f4f5d-3d91-499a-b86b-5761cafb8663 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 08:34:37.741172 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:37.741172 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:37.741172 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:37.741172 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:37.741172 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:37.741172 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:37.741172 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:37.741172 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:37.741172 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:37.741172 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:37.741172 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:37.741172 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:34:37.741172 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:34:37.741172 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:34:37.741172 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:34:37.743056 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:37.743056 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:37.743056 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:34:37.743056 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:34:37.743056 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:34:37.743056 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:34:37.743056 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:37.743056 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:37.743056 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:37.743056 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:37.743056 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:37.743056 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:37.743056 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:37.743056 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:37.743056 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:37.743056 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:37.743056 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:37.743056 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:37.744926 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:37.744926 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:37.744926 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:34:37.744926 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:34:37.744926 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:34:37.744926 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:34:37.744926 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:34:37.744926 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:34:37.744926 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 08:34:37.744926 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:34:37.749419 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 291/588] 10.222.0.55 () {56 vars in 1102 bytes} [Wed Aug 24 08:34:37 2022] GET /identity/v3/users/neutron => generated 85 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:37.751250 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891f4f5d-3d91-499a-b86b-5761cafb8663 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:37.765014 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bca1c4-71d5-46c0-ac70-eb9eb435c521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:37.766023 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bca1c4-71d5-46c0-ac70-eb9eb435c521 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:37.766489 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bca1c4-71d5-46c0-ac70-eb9eb435c521 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:37.766951 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bca1c4-71d5-46c0-ac70-eb9eb435c521 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:37.791060 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bca1c4-71d5-46c0-ac70-eb9eb435c521 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:37.794280 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bca1c4-71d5-46c0-ac70-eb9eb435c521 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:37.827381 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-f1bca1c4-71d5-46c0-ac70-eb9eb435c521 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:34:37.827381 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:37.827381 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:37.827381 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:37.827381 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:37.827381 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:37.827381 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:37.827381 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:37.827381 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:37.827381 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:37.827381 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:37.827381 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:37.827381 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:34:37.827381 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:34:37.827381 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:34:37.827381 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:34:37.829258 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:37.829258 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:37.829258 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:37.829258 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:37.829258 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:37.829258 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:37.829258 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:37.829258 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:37.829258 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:37.829258 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:37.829258 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:37.829258 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:37.829258 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:37.829258 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:37.829258 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:37.829258 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:37.829258 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:34:37.831833 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:34:37.831833 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:34:37.831833 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:34:37.835164 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 298/589] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:34:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:37.836769 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bca1c4-71d5-46c0-ac70-eb9eb435c521 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:37.940995 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:37.963230 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891f4f5d-3d91-499a-b86b-5761cafb8663 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:38.029085 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b831af3c-dd9c-4f1c-a37e-b037dfa07051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:38.030618 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b831af3c-dd9c-4f1c-a37e-b037dfa07051 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:38.030618 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b831af3c-dd9c-4f1c-a37e-b037dfa07051 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:38.030945 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b831af3c-dd9c-4f1c-a37e-b037dfa07051 None admin] PATH_INFO: `/v3/domains` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:38.031989 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b831af3c-dd9c-4f1c-a37e-b037dfa07051 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:38.032517 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b831af3c-dd9c-4f1c-a37e-b037dfa07051 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:38.035009 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b831af3c-dd9c-4f1c-a37e-b037dfa07051 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:38.054687 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bca1c4-71d5-46c0-ac70-eb9eb435c521 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:38.057513 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 292/590] 10.222.0.55 () {56 vars in 1109 bytes} [Wed Aug 24 08:34:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:38.059236 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b831af3c-dd9c-4f1c-a37e-b037dfa07051 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:38.122814 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd97016f-d6a3-4ee9-b9b0-cc2f84165fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:38.124259 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd97016f-d6a3-4ee9-b9b0-cc2f84165fd4 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:38.125111 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd97016f-d6a3-4ee9-b9b0-cc2f84165fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:38.125808 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd97016f-d6a3-4ee9-b9b0-cc2f84165fd4 None admin] PATH_INFO: `/v3/users` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:38.127376 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd97016f-d6a3-4ee9-b9b0-cc2f84165fd4 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:38.128426 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd97016f-d6a3-4ee9-b9b0-cc2f84165fd4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:38.131436 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd97016f-d6a3-4ee9-b9b0-cc2f84165fd4 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:38.154743 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:38.168196 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 299/591] 10.222.0.55 () {56 vars in 1139 bytes} [Wed Aug 24 08:34:37 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:38.171595 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd97016f-d6a3-4ee9-b9b0-cc2f84165fd4 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:38.247859 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b831af3c-dd9c-4f1c-a37e-b037dfa07051 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:38.309629 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963f9ee4-6450-472c-b3bf-98a2aa221791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:38.311297 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963f9ee4-6450-472c-b3bf-98a2aa221791 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:38.311297 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963f9ee4-6450-472c-b3bf-98a2aa221791 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:38.311598 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963f9ee4-6450-472c-b3bf-98a2aa221791 None admin] PATH_INFO: `/v3/domains` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:38.312752 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963f9ee4-6450-472c-b3bf-98a2aa221791 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:38.313263 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963f9ee4-6450-472c-b3bf-98a2aa221791 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:38.315610 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963f9ee4-6450-472c-b3bf-98a2aa221791 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:38.334068 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 293/592] 10.222.0.55 () {56 vars in 1109 bytes} [Wed Aug 24 08:34:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:38.334940 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963f9ee4-6450-472c-b3bf-98a2aa221791 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:38.385482 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd97016f-d6a3-4ee9-b9b0-cc2f84165fd4 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:38.389751 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:38.451612 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963f9ee4-6450-472c-b3bf-98a2aa221791 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:38.454730 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb4e771-4d0c-4999-b1e3-12f5194a9ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:38.456643 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb4e771-4d0c-4999-b1e3-12f5194a9ea3 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:38.457334 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb4e771-4d0c-4999-b1e3-12f5194a9ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:38.458022 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb4e771-4d0c-4999-b1e3-12f5194a9ea3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:38.462430 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:38.472434 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb4e771-4d0c-4999-b1e3-12f5194a9ea3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:38.484852 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb4e771-4d0c-4999-b1e3-12f5194a9ea3 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:38.490847 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe766e5c-d8a4-499b-b56d-6d492374810e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:38.491808 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe766e5c-d8a4-499b-b56d-6d492374810e None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:38.492319 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe766e5c-d8a4-499b-b56d-6d492374810e None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:38.492889 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe766e5c-d8a4-499b-b56d-6d492374810e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:38.499127 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-deb4e771-4d0c-4999-b1e3-12f5194a9ea3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:34:38.499127 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:38.499127 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:38.499127 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:38.499127 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:38.499127 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:38.499127 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:38.499127 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:38.499127 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:38.499127 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:38.499127 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:38.499127 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:38.499127 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:34:38.499127 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:34:38.499127 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:34:38.500071 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:34:38.500071 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:38.500071 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:38.500071 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:38.500071 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:38.500071 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:38.500071 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:38.500071 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:38.500071 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:38.500071 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:38.500071 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:38.500071 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:38.500071 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:38.500071 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:38.500071 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:38.500071 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:38.500071 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:38.500992 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:34:38.500992 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:34:38.500992 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:34:38.500992 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:34:38.500992 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:34:38.500992 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:34:38.500992 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:34:38.500992 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:34:38.501962 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 300/593] 10.222.0.55 () {56 vars in 1111 bytes} [Wed Aug 24 08:34:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:38.503908 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb4e771-4d0c-4999-b1e3-12f5194a9ea3 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:38.512505 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe766e5c-d8a4-499b-b56d-6d492374810e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:38.514419 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe766e5c-d8a4-499b-b56d-6d492374810e None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:38.532372 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-fe766e5c-d8a4-499b-b56d-6d492374810e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:34:38.532372 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:38.532372 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:38.532372 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:38.532372 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:38.532372 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:38.532372 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:38.532372 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:38.532372 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:38.532372 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:38.532372 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:38.532372 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:38.532372 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:34:38.532372 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:34:38.532372 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:34:38.532372 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:34:38.533419 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:38.533419 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:38.533419 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:38.533419 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:38.533419 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:38.533419 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:38.533419 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:38.533419 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:38.533419 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:38.533419 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:38.533419 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:38.533419 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:38.533419 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:38.533419 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:38.533419 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:38.533419 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:38.533419 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:34:38.534341 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:34:38.534341 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:34:38.534341 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:34:38.536356 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 294/594] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:34:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:38.538002 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe766e5c-d8a4-499b-b56d-6d492374810e None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:38.705665 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb4e771-4d0c-4999-b1e3-12f5194a9ea3 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:38.769100 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eca3617-310b-4d91-bf7e-0d0548192524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:38.770345 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca3617-310b-4d91-bf7e-0d0548192524 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:38.770918 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca3617-310b-4d91-bf7e-0d0548192524 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:38.771479 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca3617-310b-4d91-bf7e-0d0548192524 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:38.779936 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe766e5c-d8a4-499b-b56d-6d492374810e None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:38.785085 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eca3617-310b-4d91-bf7e-0d0548192524 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:38.786549 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eca3617-310b-4d91-bf7e-0d0548192524 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:38.800658 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-3eca3617-310b-4d91-bf7e-0d0548192524 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 08:34:38.800658 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:38.800658 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:38.800658 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:38.800658 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:38.800658 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:38.800658 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:38.800658 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:38.800658 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:38.800658 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:38.800658 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:38.800658 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:38.800658 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:34:38.800658 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:34:38.800658 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:34:38.800658 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:34:38.801924 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:38.801924 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:38.801924 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:34:38.801924 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:34:38.801924 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:34:38.801924 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:34:38.801924 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:38.801924 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:38.801924 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:38.801924 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:38.801924 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:38.801924 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:38.801924 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:38.801924 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:38.801924 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:38.801924 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:38.801924 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:38.801924 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:38.803761 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:38.803761 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:38.803761 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:34:38.803761 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:34:38.803761 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:34:38.803761 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:34:38.803761 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:34:38.803761 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:34:38.803761 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 08:34:38.803761 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:34:38.803761 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 301/595] 10.222.0.55 () {56 vars in 1099 bytes} [Wed Aug 24 08:34:38 2022] GET /identity/v3/users/cinder => generated 84 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:38.803761 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eca3617-310b-4d91-bf7e-0d0548192524 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:38.832029 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe352aa2-4e01-44f3-bd92-20a0a3a68d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:38.832947 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe352aa2-4e01-44f3-bd92-20a0a3a68d18 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:38.833415 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe352aa2-4e01-44f3-bd92-20a0a3a68d18 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:38.833878 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe352aa2-4e01-44f3-bd92-20a0a3a68d18 None admin] PATH_INFO: `/v3/projects` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:38.834809 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe352aa2-4e01-44f3-bd92-20a0a3a68d18 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:38.835275 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe352aa2-4e01-44f3-bd92-20a0a3a68d18 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:38.837125 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe352aa2-4e01-44f3-bd92-20a0a3a68d18 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:38.837981 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:38.861645 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 295/596] 10.222.0.55 () {56 vars in 1148 bytes} [Wed Aug 24 08:34:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:38.863278 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcafa6bb-1e52-4309-9931-926ac9ab2c9b None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:38.863831 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcafa6bb-1e52-4309-9931-926ac9ab2c9b None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:38.864262 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcafa6bb-1e52-4309-9931-926ac9ab2c9b None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:38.866923 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 296/597] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:34:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:34:38.868260 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcafa6bb-1e52-4309-9931-926ac9ab2c9b None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:38.903570 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:38.952515 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eca3617-310b-4d91-bf7e-0d0548192524 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:38.994537 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f065b6-cec3-4485-b4ea-0608b0118088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:38.995066 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f065b6-cec3-4485-b4ea-0608b0118088 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:38.995342 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f065b6-cec3-4485-b4ea-0608b0118088 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:38.995617 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f065b6-cec3-4485-b4ea-0608b0118088 None admin] PATH_INFO: `/v3/domains` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:38.996326 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f065b6-cec3-4485-b4ea-0608b0118088 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:38.996697 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f065b6-cec3-4485-b4ea-0608b0118088 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:38.998130 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f065b6-cec3-4485-b4ea-0608b0118088 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:39.012684 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcafa6bb-1e52-4309-9931-926ac9ab2c9b None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:39.022475 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 302/598] 10.222.0.55 () {56 vars in 1109 bytes} [Wed Aug 24 08:34:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:39.022475 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f065b6-cec3-4485-b4ea-0608b0118088 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:39.060762 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33898ce-d234-4980-b6f1-ce49c0ba8edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:39.061367 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33898ce-d234-4980-b6f1-ce49c0ba8edb None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:39.061646 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33898ce-d234-4980-b6f1-ce49c0ba8edb None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:39.061927 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33898ce-d234-4980-b6f1-ce49c0ba8edb None admin] PATH_INFO: `/v3/users` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:39.062729 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33898ce-d234-4980-b6f1-ce49c0ba8edb None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:39.063041 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33898ce-d234-4980-b6f1-ce49c0ba8edb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:39.064584 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33898ce-d234-4980-b6f1-ce49c0ba8edb None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:39.088960 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 297/599] 10.222.0.55 () {56 vars in 1137 bytes} [Wed Aug 24 08:34:38 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:39.088960 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0074ab5f-95eb-41b4-9ef1-6cd2f44bf13f None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:39.088960 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0074ab5f-95eb-41b4-9ef1-6cd2f44bf13f None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:39.088960 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0074ab5f-95eb-41b4-9ef1-6cd2f44bf13f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:39.108507 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-0074ab5f-95eb-41b4-9ef1-6cd2f44bf13f None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:39.109764 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-0074ab5f-95eb-41b4-9ef1-6cd2f44bf13f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:39.134581 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:39.156080 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f065b6-cec3-4485-b4ea-0608b0118088 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:39.187599 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 298/600] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:34:39.188475 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0074ab5f-95eb-41b4-9ef1-6cd2f44bf13f None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:39.214624 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:39.217277 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742118f9-5e61-42a3-8e30-eb1384cd4e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:39.218140 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742118f9-5e61-42a3-8e30-eb1384cd4e8a None admin] REQUEST_METHOD: `PUT` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:39.218580 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742118f9-5e61-42a3-8e30-eb1384cd4e8a None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:39.218958 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742118f9-5e61-42a3-8e30-eb1384cd4e8a None admin] PATH_INFO: `/v3/projects/420b18725f2342159646fb86438318fd/users/5512043ca9a7421b821741ca88d50b07/roles/bab90e02eebc473e9b911c8660ed9b76` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:39.238942 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742118f9-5e61-42a3-8e30-eb1384cd4e8a None admin] RBAC: Authorizing `identity:create_grant(project_id=420b18725f2342159646fb86438318fd, user_id=5512043ca9a7421b821741ca88d50b07, role_id=bab90e02eebc473e9b911c8660ed9b76)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:39.240699 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742118f9-5e61-42a3-8e30-eb1384cd4e8a None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:39.316888 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 303/601] 10.222.0.55 () {58 vars in 1439 bytes} [Wed Aug 24 08:34:39 2022] PUT /identity/v3/projects/420b18725f2342159646fb86438318fd/users/5512043ca9a7421b821741ca88d50b07/roles/bab90e02eebc473e9b911c8660ed9b76 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:34:39.318349 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742118f9-5e61-42a3-8e30-eb1384cd4e8a None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:39.372493 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0074ab5f-95eb-41b4-9ef1-6cd2f44bf13f None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:39.454067 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c41a7f3-d410-44c9-964f-76a03d6e7bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:39.456619 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c41a7f3-d410-44c9-964f-76a03d6e7bca None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:39.457396 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c41a7f3-d410-44c9-964f-76a03d6e7bca None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:39.458117 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c41a7f3-d410-44c9-964f-76a03d6e7bca None admin] PATH_INFO: `/v3/projects/service` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:39.475917 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c41a7f3-d410-44c9-964f-76a03d6e7bca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:39.479068 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c41a7f3-d410-44c9-964f-76a03d6e7bca None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:39.502037 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-6c41a7f3-d410-44c9-964f-76a03d6e7bca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:34:39.502037 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:39.502037 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:39.502037 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:39.502037 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:39.502037 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:39.502037 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:39.502037 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:39.502037 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:39.502037 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:39.502037 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:39.502037 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:39.502037 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:34:39.502037 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:34:39.502037 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:34:39.504017 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:34:39.504017 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:39.504017 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:39.504017 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:39.504017 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:39.504017 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:39.504017 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:39.504017 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:39.504017 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:39.504017 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:39.504017 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:39.504017 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:39.504017 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:39.504017 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:39.504017 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:39.504017 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:39.504017 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:39.505453 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:34:39.505453 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:34:39.505453 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:34:39.505453 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:34:39.505453 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:34:39.505453 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:34:39.505453 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:34:39.505453 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:34:39.513007 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 299/602] 10.222.0.55 () {56 vars in 1111 bytes} [Wed Aug 24 08:34:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:39.513007 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fcf735-90d6-4339-a36c-9e7a3a3c70a5 None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:39.513007 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fcf735-90d6-4339-a36c-9e7a3a3c70a5 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:39.513007 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fcf735-90d6-4339-a36c-9e7a3a3c70a5 None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:39.513007 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 300/603] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:34:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:34:39.514366 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fcf735-90d6-4339-a36c-9e7a3a3c70a5 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:39.526823 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742118f9-5e61-42a3-8e30-eb1384cd4e8a None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:39.564489 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:39.573667 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c70b37-8b56-492e-ade3-58b763d516b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:39.575323 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c70b37-8b56-492e-ade3-58b763d516b7 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:39.575869 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c70b37-8b56-492e-ade3-58b763d516b7 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:39.576577 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c70b37-8b56-492e-ade3-58b763d516b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:39.595335 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c70b37-8b56-492e-ade3-58b763d516b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:39.597995 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c70b37-8b56-492e-ade3-58b763d516b7 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:39.600485 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:39.626756 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-e2c70b37-8b56-492e-ade3-58b763d516b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:34:39.626756 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:39.626756 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:39.626756 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:39.626756 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:39.626756 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:39.626756 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:39.626756 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:39.626756 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:39.626756 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:39.626756 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:39.626756 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:39.626756 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:34:39.626756 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:34:39.626756 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:34:39.626756 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:34:39.627772 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:39.627772 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:39.627772 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:39.627772 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:39.627772 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:39.627772 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:39.627772 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:39.627772 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:39.627772 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:39.627772 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:39.627772 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:39.627772 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:39.627772 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:39.627772 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:39.627772 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:39.627772 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:39.627772 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:34:39.628963 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:34:39.628963 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:34:39.628963 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:34:39.628963 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 304/604] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:34:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:39.630470 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c70b37-8b56-492e-ade3-58b763d516b7 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:39.643795 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fcf735-90d6-4339-a36c-9e7a3a3c70a5 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:39.701629 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877cb0c2-189d-4425-8eae-5ca1ffae814a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:39.702674 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877cb0c2-189d-4425-8eae-5ca1ffae814a None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:39.703141 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877cb0c2-189d-4425-8eae-5ca1ffae814a None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:39.703599 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877cb0c2-189d-4425-8eae-5ca1ffae814a None admin] PATH_INFO: `/v3/roles/service` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:39.722655 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877cb0c2-189d-4425-8eae-5ca1ffae814a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:34:39.731856 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-877cb0c2-189d-4425-8eae-5ca1ffae814a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:34:39.731856 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:39.731856 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:39.731856 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:39.731856 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:39.731856 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:39.731856 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:39.731856 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:39.731856 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:39.731856 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:39.731856 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:39.731856 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:39.731856 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:34:39.731856 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:34:39.731856 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:34:39.731856 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:34:39.732899 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:34:39.732899 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:34:39.732899 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:34:39.732899 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:34:39.732899 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:39.732899 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:39.732899 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:39.732899 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:39.732899 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:39.732899 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:39.732899 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:39.732899 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:39.732899 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:39.732899 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:39.732899 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:39.732899 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:39.738685 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:39.738685 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:39.738685 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:39.738685 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:39.738685 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:34:39.738685 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:34:39.738685 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:34:39.738685 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:34:39.738685 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:34:39.738685 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:34:39.738685 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:34:39.738685 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:34:39.738685 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 301/605] 10.222.0.55 () {56 vars in 1102 bytes} [Wed Aug 24 08:34:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:39.738685 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beec8663-5780-4592-a2fb-e0cd5ed21f77 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:39.739529 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beec8663-5780-4592-a2fb-e0cd5ed21f77 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:39.739529 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beec8663-5780-4592-a2fb-e0cd5ed21f77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:39.768709 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-beec8663-5780-4592-a2fb-e0cd5ed21f77 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:39.769079 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c70b37-8b56-492e-ade3-58b763d516b7 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:39.771063 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-beec8663-5780-4592-a2fb-e0cd5ed21f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:39.809403 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01fff03-550d-4b45-98e0-48a8d63e15d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:39.810679 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01fff03-550d-4b45-98e0-48a8d63e15d7 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:39.810955 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01fff03-550d-4b45-98e0-48a8d63e15d7 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:39.811224 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01fff03-550d-4b45-98e0-48a8d63e15d7 None admin] PATH_INFO: `/v3/projects` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:39.811930 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01fff03-550d-4b45-98e0-48a8d63e15d7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:39.812229 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01fff03-550d-4b45-98e0-48a8d63e15d7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:39.813722 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01fff03-550d-4b45-98e0-48a8d63e15d7 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:39.834510 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 305/606] 10.222.0.55 () {56 vars in 1148 bytes} [Wed Aug 24 08:34:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:39.834510 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01fff03-550d-4b45-98e0-48a8d63e15d7 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:39.873042 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 302/607] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:34:39.873911 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beec8663-5780-4592-a2fb-e0cd5ed21f77 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:39.910362 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:39.969096 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:40.024665 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01fff03-550d-4b45-98e0-48a8d63e15d7 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:40.030363 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beec8663-5780-4592-a2fb-e0cd5ed21f77 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:40.063102 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dfe300-8961-4f74-a5bc-26d66fa628c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:40.063787 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dfe300-8961-4f74-a5bc-26d66fa628c5 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:40.064066 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dfe300-8961-4f74-a5bc-26d66fa628c5 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:40.064342 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dfe300-8961-4f74-a5bc-26d66fa628c5 None admin] PATH_INFO: `/v3/domains` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:40.065358 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dfe300-8961-4f74-a5bc-26d66fa628c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:40.065358 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dfe300-8961-4f74-a5bc-26d66fa628c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:40.066701 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dfe300-8961-4f74-a5bc-26d66fa628c5 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:40.082260 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 306/608] 10.222.0.55 () {56 vars in 1109 bytes} [Wed Aug 24 08:34:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:40.083114 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dfe300-8961-4f74-a5bc-26d66fa628c5 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:40.112895 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7f67ff-e5e9-4496-ad9e-7eeeece6f5c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:40.114551 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7f67ff-e5e9-4496-ad9e-7eeeece6f5c6 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:40.115496 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7f67ff-e5e9-4496-ad9e-7eeeece6f5c6 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:40.117109 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7f67ff-e5e9-4496-ad9e-7eeeece6f5c6 None admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:40.118948 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7f67ff-e5e9-4496-ad9e-7eeeece6f5c6 None admin] RBAC: Adding query filter params (name=service) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:40.120219 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7f67ff-e5e9-4496-ad9e-7eeeece6f5c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:40.123703 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7f67ff-e5e9-4496-ad9e-7eeeece6f5c6 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:40.139932 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 303/609] 10.222.0.55 () {56 vars in 1103 bytes} [Wed Aug 24 08:34:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:40.141032 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7f67ff-e5e9-4496-ad9e-7eeeece6f5c6 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:40.225425 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dfe300-8961-4f74-a5bc-26d66fa628c5 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:40.228190 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:40.275666 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309c7398-f74e-4e3f-b94b-88fa18075185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:40.276346 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309c7398-f74e-4e3f-b94b-88fa18075185 None admin] REQUEST_METHOD: `PUT` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:40.276667 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309c7398-f74e-4e3f-b94b-88fa18075185 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:40.277012 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309c7398-f74e-4e3f-b94b-88fa18075185 None admin] PATH_INFO: `/v3/projects/420b18725f2342159646fb86438318fd/users/28b58388d1664510b0b507323bca5ae8/roles/bab90e02eebc473e9b911c8660ed9b76` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:40.277192 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7f67ff-e5e9-4496-ad9e-7eeeece6f5c6 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:40.295394 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309c7398-f74e-4e3f-b94b-88fa18075185 None admin] RBAC: Authorizing `identity:create_grant(project_id=420b18725f2342159646fb86438318fd, user_id=28b58388d1664510b0b507323bca5ae8, role_id=bab90e02eebc473e9b911c8660ed9b76)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:40.296901 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309c7398-f74e-4e3f-b94b-88fa18075185 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:40.320764 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:40.337059 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d76c2a-02f5-4f9c-8afe-fb09277c0c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:40.337859 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d76c2a-02f5-4f9c-8afe-fb09277c0c69 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:40.338225 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d76c2a-02f5-4f9c-8afe-fb09277c0c69 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:40.338602 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d76c2a-02f5-4f9c-8afe-fb09277c0c69 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:40.348615 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d76c2a-02f5-4f9c-8afe-fb09277c0c69 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:40.350232 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d76c2a-02f5-4f9c-8afe-fb09277c0c69 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:40.363490 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-05d76c2a-02f5-4f9c-8afe-fb09277c0c69 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:34:40.363490 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:40.363490 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:40.363490 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:40.363490 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:40.363490 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:40.363490 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:40.363490 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:40.363490 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:40.363490 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:40.363490 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:40.363490 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:40.363490 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:34:40.363490 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:34:40.363490 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:34:40.365392 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:34:40.365392 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:40.365392 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:40.365392 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:40.365392 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:40.365392 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:40.365392 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:40.365392 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:40.365392 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:40.365392 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:40.365392 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:40.365392 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:40.365392 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:40.365392 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:40.365392 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:40.365392 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:40.365392 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:40.367505 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:34:40.367505 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:34:40.367505 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:34:40.367505 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:34:40.367505 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:34:40.367505 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:34:40.367505 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:34:40.367505 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:34:40.367505 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 304/610] 10.222.0.55 () {56 vars in 1111 bytes} [Wed Aug 24 08:34:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:40.367505 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d76c2a-02f5-4f9c-8afe-fb09277c0c69 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:40.422722 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 307/611] 10.222.0.55 () {58 vars in 1439 bytes} [Wed Aug 24 08:34:40 2022] PUT /identity/v3/projects/420b18725f2342159646fb86438318fd/users/28b58388d1664510b0b507323bca5ae8/roles/bab90e02eebc473e9b911c8660ed9b76 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:34:40.424469 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309c7398-f74e-4e3f-b94b-88fa18075185 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:40.544621 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d76c2a-02f5-4f9c-8afe-fb09277c0c69 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:40.623611 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d139a81-4aec-4878-a326-ccf07e84e345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:40.625275 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d139a81-4aec-4878-a326-ccf07e84e345 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:40.625786 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d139a81-4aec-4878-a326-ccf07e84e345 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:40.627025 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d139a81-4aec-4878-a326-ccf07e84e345 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:40.634764 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309c7398-f74e-4e3f-b94b-88fa18075185 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:40.655746 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d139a81-4aec-4878-a326-ccf07e84e345 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:40.657287 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d139a81-4aec-4878-a326-ccf07e84e345 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:40.674338 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-6d139a81-4aec-4878-a326-ccf07e84e345 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:34:40.674338 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:40.674338 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:40.674338 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:40.674338 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:40.674338 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:40.674338 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:40.674338 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:40.674338 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:40.674338 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:40.674338 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:40.674338 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:40.674338 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:34:40.674338 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:34:40.674338 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:34:40.674338 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:34:40.675458 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:40.675458 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:40.675458 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:40.675458 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:40.675458 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:40.675458 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:40.675458 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:40.675458 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:40.675458 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:40.675458 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:40.675458 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:40.675458 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:40.675458 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:40.675458 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:40.675458 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:40.675458 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:40.675458 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:34:40.676925 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:34:40.676925 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:34:40.676925 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:34:40.676925 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 305/612] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:34:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:40.676925 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d139a81-4aec-4878-a326-ccf07e84e345 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:40.695605 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21305d6a-d8c7-4df4-87af-461eccd00df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:40.696259 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21305d6a-d8c7-4df4-87af-461eccd00df9 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:40.696529 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21305d6a-d8c7-4df4-87af-461eccd00df9 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:40.697336 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21305d6a-d8c7-4df4-87af-461eccd00df9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:40.713206 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21305d6a-d8c7-4df4-87af-461eccd00df9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:34:40.718320 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-21305d6a-d8c7-4df4-87af-461eccd00df9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:34:40.718320 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:40.718320 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:40.718320 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:40.718320 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:40.718320 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:40.718320 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:40.718320 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:40.718320 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:40.718320 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:40.718320 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:40.718320 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:40.718320 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:34:40.718320 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:34:40.718320 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:34:40.718320 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:34:40.719463 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:34:40.719463 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:34:40.719463 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:34:40.719463 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:34:40.719463 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:40.719463 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:40.719463 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:40.719463 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:40.719463 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:40.719463 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:40.719463 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:40.719463 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:40.719463 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:40.719463 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:40.719463 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:40.719463 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:40.720381 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:40.720381 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:40.720381 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:40.720381 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:40.720381 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:34:40.720381 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:34:40.720381 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:34:40.720381 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:34:40.720381 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:34:40.720381 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:34:40.720381 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:34:40.720381 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:34:40.722948 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 308/613] 10.222.0.55 () {56 vars in 1102 bytes} [Wed Aug 24 08:34:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:40.722948 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21305d6a-d8c7-4df4-87af-461eccd00df9 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:40.726912 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:40.745332 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:40.844289 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d139a81-4aec-4878-a326-ccf07e84e345 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:40.873450 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21305d6a-d8c7-4df4-87af-461eccd00df9 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:40.888045 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935ab427-17b9-4b45-a02d-498c2300f443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:40.888657 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935ab427-17b9-4b45-a02d-498c2300f443 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:40.889433 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935ab427-17b9-4b45-a02d-498c2300f443 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:40.889991 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935ab427-17b9-4b45-a02d-498c2300f443 None admin] PATH_INFO: `/v3/projects` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:40.891174 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935ab427-17b9-4b45-a02d-498c2300f443 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:40.892345 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935ab427-17b9-4b45-a02d-498c2300f443 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:40.893977 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935ab427-17b9-4b45-a02d-498c2300f443 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:40.910113 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 306/614] 10.222.0.55 () {56 vars in 1148 bytes} [Wed Aug 24 08:34:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:40.910987 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935ab427-17b9-4b45-a02d-498c2300f443 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:40.953060 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ffa9ce-3b09-41eb-a765-d55223f529c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:40.954398 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ffa9ce-3b09-41eb-a765-d55223f529c2 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:40.954896 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ffa9ce-3b09-41eb-a765-d55223f529c2 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:40.956011 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ffa9ce-3b09-41eb-a765-d55223f529c2 None admin] PATH_INFO: `/v3/domains` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:40.957439 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ffa9ce-3b09-41eb-a765-d55223f529c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:40.957987 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ffa9ce-3b09-41eb-a765-d55223f529c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:40.962145 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ffa9ce-3b09-41eb-a765-d55223f529c2 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:40.991296 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 309/615] 10.222.0.55 () {56 vars in 1109 bytes} [Wed Aug 24 08:34:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:40.992886 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ffa9ce-3b09-41eb-a765-d55223f529c2 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:41.061415 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:41.077971 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935ab427-17b9-4b45-a02d-498c2300f443 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:41.114030 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:41.150721 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc70544-f22e-4e10-9c6c-0eb51df7f550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:41.151825 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc70544-f22e-4e10-9c6c-0eb51df7f550 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:41.152400 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc70544-f22e-4e10-9c6c-0eb51df7f550 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:41.153025 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc70544-f22e-4e10-9c6c-0eb51df7f550 None admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:41.154447 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc70544-f22e-4e10-9c6c-0eb51df7f550 None admin] RBAC: Adding query filter params (name=service) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:41.155430 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc70544-f22e-4e10-9c6c-0eb51df7f550 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:41.158274 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc70544-f22e-4e10-9c6c-0eb51df7f550 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:41.165291 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ffa9ce-3b09-41eb-a765-d55223f529c2 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:41.176425 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 307/616] 10.222.0.55 () {56 vars in 1103 bytes} [Wed Aug 24 08:34:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:41.178345 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc70544-f22e-4e10-9c6c-0eb51df7f550 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:41.251391 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16e2c3c-bcd7-4c81-8d48-b48d3d672732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:41.253150 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16e2c3c-bcd7-4c81-8d48-b48d3d672732 None admin] REQUEST_METHOD: `PUT` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:41.253906 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16e2c3c-bcd7-4c81-8d48-b48d3d672732 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:41.254498 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16e2c3c-bcd7-4c81-8d48-b48d3d672732 None admin] PATH_INFO: `/v3/projects/420b18725f2342159646fb86438318fd/users/fbbb9ac667f04c35a63b2689df18efed/roles/bab90e02eebc473e9b911c8660ed9b76` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:41.291697 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16e2c3c-bcd7-4c81-8d48-b48d3d672732 None admin] RBAC: Authorizing `identity:create_grant(project_id=420b18725f2342159646fb86438318fd, user_id=fbbb9ac667f04c35a63b2689df18efed, role_id=bab90e02eebc473e9b911c8660ed9b76)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:41.295189 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16e2c3c-bcd7-4c81-8d48-b48d3d672732 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:41.327878 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc70544-f22e-4e10-9c6c-0eb51df7f550 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:41.366963 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec2e95e-19c1-4c37-ae36-51b0c7092666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:41.368007 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec2e95e-19c1-4c37-ae36-51b0c7092666 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:41.368510 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec2e95e-19c1-4c37-ae36-51b0c7092666 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:41.369079 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec2e95e-19c1-4c37-ae36-51b0c7092666 None admin] PATH_INFO: `/v3/users/swift` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:41.378634 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec2e95e-19c1-4c37-ae36-51b0c7092666 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:41.380502 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec2e95e-19c1-4c37-ae36-51b0c7092666 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:41.394720 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-eec2e95e-19c1-4c37-ae36-51b0c7092666 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 08:34:41.394720 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:41.394720 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:41.394720 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:41.394720 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:41.394720 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:41.394720 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:41.394720 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:41.394720 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:41.394720 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:41.394720 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:41.394720 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:41.394720 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:34:41.394720 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:34:41.394720 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:34:41.394720 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:34:41.395763 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:41.395763 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:41.395763 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:34:41.395763 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:34:41.395763 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:34:41.395763 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:34:41.395763 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:41.395763 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:41.395763 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:41.395763 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:41.395763 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:41.395763 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:41.395763 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:41.395763 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:41.395763 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:41.395763 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:41.395763 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:41.395763 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:41.396953 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:41.396953 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:41.396953 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:34:41.396953 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:34:41.396953 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:34:41.396953 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:34:41.396953 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:34:41.396953 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:34:41.396953 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 08:34:41.396953 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:34:41.397526 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 310/617] 10.222.0.55 () {58 vars in 1439 bytes} [Wed Aug 24 08:34:40 2022] PUT /identity/v3/projects/420b18725f2342159646fb86438318fd/users/fbbb9ac667f04c35a63b2689df18efed/roles/bab90e02eebc473e9b911c8660ed9b76 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:34:41.398159 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16e2c3c-bcd7-4c81-8d48-b48d3d672732 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:41.399893 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 308/618] 10.222.0.55 () {56 vars in 1096 bytes} [Wed Aug 24 08:34:41 2022] GET /identity/v3/users/swift => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:41.408745 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec2e95e-19c1-4c37-ae36-51b0c7092666 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:41.452456 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:41.599884 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16e2c3c-bcd7-4c81-8d48-b48d3d672732 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:41.635518 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec2e95e-19c1-4c37-ae36-51b0c7092666 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:41.641234 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:41.658996 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a96ef7-9b8a-48d2-8183-6aec1f8c4b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:41.660181 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a96ef7-9b8a-48d2-8183-6aec1f8c4b3b None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:41.660747 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a96ef7-9b8a-48d2-8183-6aec1f8c4b3b None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:41.661507 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a96ef7-9b8a-48d2-8183-6aec1f8c4b3b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:41.691408 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a96ef7-9b8a-48d2-8183-6aec1f8c4b3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:41.695739 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a96ef7-9b8a-48d2-8183-6aec1f8c4b3b None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:41.699895 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ff62da-3d2e-4884-9cd7-e936fa77270f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:41.701025 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ff62da-3d2e-4884-9cd7-e936fa77270f None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:41.701511 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ff62da-3d2e-4884-9cd7-e936fa77270f None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:41.702034 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ff62da-3d2e-4884-9cd7-e936fa77270f None admin] PATH_INFO: `/v3/users` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:41.703638 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ff62da-3d2e-4884-9cd7-e936fa77270f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:41.704162 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ff62da-3d2e-4884-9cd7-e936fa77270f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:41.706936 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ff62da-3d2e-4884-9cd7-e936fa77270f None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:41.731270 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-82a96ef7-9b8a-48d2-8183-6aec1f8c4b3b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:34:41.731270 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:41.731270 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:41.731270 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:41.731270 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:41.731270 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:41.731270 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:41.731270 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:41.731270 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:41.731270 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:41.731270 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:41.731270 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:41.731270 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:34:41.731270 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:34:41.731270 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:34:41.731270 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:34:41.733138 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:41.733138 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:41.733138 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:41.733138 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:41.733138 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:41.733138 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:41.733138 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:41.733138 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:41.733138 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:41.733138 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:41.733138 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:41.733138 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:41.733138 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:41.733138 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:41.733138 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:41.733138 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:41.733138 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:34:41.734838 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:34:41.734838 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:34:41.734838 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:34:41.738929 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 311/619] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:34:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:41.755550 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a96ef7-9b8a-48d2-8183-6aec1f8c4b3b None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:41.775593 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 309/620] 10.222.0.55 () {56 vars in 1135 bytes} [Wed Aug 24 08:34:41 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:41.781638 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ff62da-3d2e-4884-9cd7-e936fa77270f None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:41.942475 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a96ef7-9b8a-48d2-8183-6aec1f8c4b3b None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:41.986483 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fead98-1485-43e8-aecb-84137036f1ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:41.987534 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fead98-1485-43e8-aecb-84137036f1ee None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:41.988027 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fead98-1485-43e8-aecb-84137036f1ee None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:41.988531 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fead98-1485-43e8-aecb-84137036f1ee None admin] PATH_INFO: `/v3/domains` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:41.989440 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fead98-1485-43e8-aecb-84137036f1ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:41.989953 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fead98-1485-43e8-aecb-84137036f1ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:41.991805 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fead98-1485-43e8-aecb-84137036f1ee None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:42.009565 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:42.017957 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 312/621] 10.222.0.55 () {56 vars in 1109 bytes} [Wed Aug 24 08:34:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:42.026240 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fead98-1485-43e8-aecb-84137036f1ee None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:42.065305 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ff62da-3d2e-4884-9cd7-e936fa77270f None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:42.070607 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:42.110323 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a3d3ce-0b24-4837-80d9-ec0990fb1001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:42.111189 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a3d3ce-0b24-4837-80d9-ec0990fb1001 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:42.111598 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a3d3ce-0b24-4837-80d9-ec0990fb1001 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:42.111962 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a3d3ce-0b24-4837-80d9-ec0990fb1001 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:42.125295 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a3d3ce-0b24-4837-80d9-ec0990fb1001 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:42.127572 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a3d3ce-0b24-4837-80d9-ec0990fb1001 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:42.145610 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fead98-1485-43e8-aecb-84137036f1ee None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:42.146709 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-84a3d3ce-0b24-4837-80d9-ec0990fb1001 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:34:42.146709 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:42.146709 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:42.146709 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:42.146709 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:42.146709 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:42.146709 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:42.146709 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:42.146709 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:42.146709 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:42.146709 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:42.146709 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:42.146709 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:34:42.146709 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:34:42.146709 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:34:42.146709 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:34:42.147613 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:42.147613 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:42.147613 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:42.147613 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:42.147613 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:42.147613 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:42.147613 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:42.147613 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:42.147613 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:42.147613 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:42.147613 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:42.147613 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:42.147613 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:42.147613 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:42.147613 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:42.147613 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:42.147613 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:34:42.148565 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:34:42.148565 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:34:42.148565 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:34:42.153392 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 310/622] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:34:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 368 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:42.155184 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa83aca-78e1-40cc-b1bd-368f21073dab None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:42.155464 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa83aca-78e1-40cc-b1bd-368f21073dab None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:42.155797 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa83aca-78e1-40cc-b1bd-368f21073dab None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:42.157729 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 311/623] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:34:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:34:42.160041 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa83aca-78e1-40cc-b1bd-368f21073dab None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:42.190039 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce855f3-d964-456d-8fbc-f2913b7491e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:42.190746 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce855f3-d964-456d-8fbc-f2913b7491e5 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:42.191024 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce855f3-d964-456d-8fbc-f2913b7491e5 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:42.191294 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce855f3-d964-456d-8fbc-f2913b7491e5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:42.211705 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce855f3-d964-456d-8fbc-f2913b7491e5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:42.213413 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce855f3-d964-456d-8fbc-f2913b7491e5 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:42.228016 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-fce855f3-d964-456d-8fbc-f2913b7491e5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 08:34:42.228016 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:42.228016 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:42.228016 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:42.228016 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:42.228016 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:42.228016 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:42.228016 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:42.228016 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:42.228016 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:42.228016 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:42.228016 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:42.228016 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:34:42.228016 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:34:42.228016 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:34:42.228016 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:34:42.229062 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:42.229062 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:42.229062 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:34:42.229062 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:34:42.229062 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:34:42.229062 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:34:42.229062 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:42.229062 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:42.229062 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:42.229062 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:42.229062 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:42.229062 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:42.229062 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:42.229062 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:42.229062 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:42.229062 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:42.229062 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:42.229062 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:42.230021 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:42.230021 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:42.230021 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:34:42.230021 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:34:42.230021 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:34:42.230021 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:34:42.230021 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:34:42.230021 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:34:42.230021 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 08:34:42.230021 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:34:42.234749 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 313/624] 10.222.0.55 () {56 vars in 1093 bytes} [Wed Aug 24 08:34:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:42.236174 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbffeb3-cbef-4c47-80d5-950077122201 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:42.236460 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbffeb3-cbef-4c47-80d5-950077122201 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:42.236766 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbffeb3-cbef-4c47-80d5-950077122201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:42.257148 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-4fbffeb3-cbef-4c47-80d5-950077122201 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:42.258496 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-4fbffeb3-cbef-4c47-80d5-950077122201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:42.333791 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa83aca-78e1-40cc-b1bd-368f21073dab None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:42.336291 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 314/625] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:34:42.337229 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbffeb3-cbef-4c47-80d5-950077122201 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:42.388958 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:42.390220 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:42.421326 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72707f32-2ecc-45d6-ae6b-7af7ff006224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:42.421968 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72707f32-2ecc-45d6-ae6b-7af7ff006224 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:42.422247 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72707f32-2ecc-45d6-ae6b-7af7ff006224 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:42.422517 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72707f32-2ecc-45d6-ae6b-7af7ff006224 None admin] PATH_INFO: `/v3/domains` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:42.423175 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72707f32-2ecc-45d6-ae6b-7af7ff006224 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:42.423471 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72707f32-2ecc-45d6-ae6b-7af7ff006224 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:42.424873 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72707f32-2ecc-45d6-ae6b-7af7ff006224 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:42.453119 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 312/626] 10.222.0.55 () {56 vars in 1109 bytes} [Wed Aug 24 08:34:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:34:42.454513 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbffeb3-cbef-4c47-80d5-950077122201 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:42.458534 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72707f32-2ecc-45d6-ae6b-7af7ff006224 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:42.495719 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d346a5ab-08b3-4b49-b834-ec49c3482a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:42.496531 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d346a5ab-08b3-4b49-b834-ec49c3482a85 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:42.496834 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d346a5ab-08b3-4b49-b834-ec49c3482a85 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:42.497105 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d346a5ab-08b3-4b49-b834-ec49c3482a85 None admin] PATH_INFO: `/v3/users` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:42.497916 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d346a5ab-08b3-4b49-b834-ec49c3482a85 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:42.498224 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d346a5ab-08b3-4b49-b834-ec49c3482a85 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:42.499691 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d346a5ab-08b3-4b49-b834-ec49c3482a85 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:42.528886 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 315/627] 10.222.0.55 () {56 vars in 1133 bytes} [Wed Aug 24 08:34:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:42.530883 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d346a5ab-08b3-4b49-b834-ec49c3482a85 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:42.654891 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d346a5ab-08b3-4b49-b834-ec49c3482a85 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:42.665233 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:42.707341 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72707f32-2ecc-45d6-ae6b-7af7ff006224 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:42.730987 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90c37cc-3f5c-49aa-b49d-e2166d1a0823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:42.731748 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90c37cc-3f5c-49aa-b49d-e2166d1a0823 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:42.732027 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90c37cc-3f5c-49aa-b49d-e2166d1a0823 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:42.732385 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90c37cc-3f5c-49aa-b49d-e2166d1a0823 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:42.772565 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2b62d0-7b5e-41d6-ab0b-9d785054a5bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:42.773590 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2b62d0-7b5e-41d6-ab0b-9d785054a5bb None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:42.774220 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2b62d0-7b5e-41d6-ab0b-9d785054a5bb None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:42.774693 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2b62d0-7b5e-41d6-ab0b-9d785054a5bb None admin] PATH_INFO: `/v3/projects/service` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:42.787557 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2b62d0-7b5e-41d6-ab0b-9d785054a5bb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:42.790611 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2b62d0-7b5e-41d6-ab0b-9d785054a5bb None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:42.812878 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90c37cc-3f5c-49aa-b49d-e2166d1a0823 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:42.814343 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90c37cc-3f5c-49aa-b49d-e2166d1a0823 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:42.814904 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-2e2b62d0-7b5e-41d6-ab0b-9d785054a5bb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:34:42.814904 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:42.814904 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:42.814904 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:42.814904 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:42.814904 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:42.814904 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:42.814904 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:42.814904 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:42.814904 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:42.814904 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:42.814904 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:42.814904 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:34:42.814904 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:34:42.814904 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:34:42.816792 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:34:42.816792 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:42.816792 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:42.816792 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:42.816792 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:42.816792 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:42.816792 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:42.816792 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:42.816792 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:42.816792 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:42.816792 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:42.816792 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:42.816792 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:42.816792 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:42.816792 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:42.816792 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:42.816792 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:42.818676 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:34:42.818676 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:34:42.818676 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:34:42.818676 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:34:42.818676 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:34:42.818676 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:34:42.818676 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:34:42.818676 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:34:42.818676 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 313/628] 10.222.0.55 () {56 vars in 1111 bytes} [Wed Aug 24 08:34:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 360 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:34:42.819654 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07ba940-0059-4519-a518-4e5b3fce53be None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:42.819654 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07ba940-0059-4519-a518-4e5b3fce53be None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:42.819654 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07ba940-0059-4519-a518-4e5b3fce53be None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:42.828108 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 314/629] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:34:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:34:42.828108 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07ba940-0059-4519-a518-4e5b3fce53be None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:42.904662 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:42.956904 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:42.987346 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 316/630] 10.222.0.55 () {58 vars in 1282 bytes} [Wed Aug 24 08:34:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 08:34:42.987346 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90c37cc-3f5c-49aa-b49d-e2166d1a0823 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:43.041657 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07ba940-0059-4519-a518-4e5b3fce53be None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:43.104491 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a28b6b-085e-495b-8f14-e76caadf2bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:43.105642 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a28b6b-085e-495b-8f14-e76caadf2bbc None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:43.106610 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a28b6b-085e-495b-8f14-e76caadf2bbc None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:43.107058 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a28b6b-085e-495b-8f14-e76caadf2bbc None admin] PATH_INFO: `/v3/roles/service` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:43.125910 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a28b6b-085e-495b-8f14-e76caadf2bbc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:34:43.134866 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-f0a28b6b-085e-495b-8f14-e76caadf2bbc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:34:43.134866 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:43.134866 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:43.134866 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:43.134866 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:43.134866 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:43.134866 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:43.134866 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:43.134866 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:43.134866 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:43.134866 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:43.134866 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:43.134866 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:34:43.134866 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:34:43.134866 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:34:43.134866 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:34:43.137319 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:34:43.137319 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:34:43.137319 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:34:43.137319 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:34:43.137319 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:43.137319 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:43.137319 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:43.137319 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:43.137319 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:43.137319 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:43.137319 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:43.137319 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:43.137319 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:43.137319 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:43.137319 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:43.137319 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:43.140067 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:43.140067 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:43.140067 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:43.140067 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:43.140067 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:34:43.140067 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:34:43.140067 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:34:43.140067 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:34:43.140067 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:34:43.140067 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:34:43.140067 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:34:43.140067 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:34:43.140067 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 315/631] 10.222.0.55 () {56 vars in 1102 bytes} [Wed Aug 24 08:34:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:43.140067 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8464acff-aeb7-4a96-b414-7b166d4907d1 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:43.141234 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8464acff-aeb7-4a96-b414-7b166d4907d1 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:43.141234 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8464acff-aeb7-4a96-b414-7b166d4907d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:43.162488 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90c37cc-3f5c-49aa-b49d-e2166d1a0823 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:43.166304 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-8464acff-aeb7-4a96-b414-7b166d4907d1 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:43.167665 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-8464acff-aeb7-4a96-b414-7b166d4907d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:43.215087 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce13e067-b872-429e-bacd-067f7a563a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:43.216172 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce13e067-b872-429e-bacd-067f7a563a83 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:43.216747 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce13e067-b872-429e-bacd-067f7a563a83 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:43.217305 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce13e067-b872-429e-bacd-067f7a563a83 None admin] PATH_INFO: `/v3/projects` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:43.218218 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce13e067-b872-429e-bacd-067f7a563a83 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:43.218735 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce13e067-b872-429e-bacd-067f7a563a83 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:43.221272 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce13e067-b872-429e-bacd-067f7a563a83 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:43.249991 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 317/632] 10.222.0.55 () {56 vars in 1148 bytes} [Wed Aug 24 08:34:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:43.249991 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce13e067-b872-429e-bacd-067f7a563a83 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:43.308743 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 316/633] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:34:43.310306 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8464acff-aeb7-4a96-b414-7b166d4907d1 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:43.341273 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:43.382672 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:43.441050 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce13e067-b872-429e-bacd-067f7a563a83 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:43.504048 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867ef742-e425-4b57-87c7-aef7101ba19d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:43.505627 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867ef742-e425-4b57-87c7-aef7101ba19d None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:43.506554 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867ef742-e425-4b57-87c7-aef7101ba19d None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:43.507237 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867ef742-e425-4b57-87c7-aef7101ba19d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:43.528253 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8464acff-aeb7-4a96-b414-7b166d4907d1 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:43.529849 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867ef742-e425-4b57-87c7-aef7101ba19d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:43.533038 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867ef742-e425-4b57-87c7-aef7101ba19d None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:43.566308 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-867ef742-e425-4b57-87c7-aef7101ba19d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:34:43.566308 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:43.566308 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:43.566308 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:43.566308 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:43.566308 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:43.566308 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:43.566308 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:43.566308 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:43.566308 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:43.566308 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:43.566308 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:43.566308 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:34:43.566308 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:34:43.566308 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:34:43.566308 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:34:43.568154 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:43.568154 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:43.568154 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:43.568154 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:43.568154 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:43.568154 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:43.568154 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:43.568154 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:43.568154 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:43.568154 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:43.568154 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:43.568154 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:43.568154 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:43.568154 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:43.568154 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:43.568154 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:43.568154 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:34:43.570063 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:34:43.570063 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:34:43.570063 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:34:43.573519 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 318/634] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:34:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:43.575530 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867ef742-e425-4b57-87c7-aef7101ba19d None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:43.607955 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13849889-93bd-4016-8509-e44e8832d385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:43.608995 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13849889-93bd-4016-8509-e44e8832d385 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:43.609448 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13849889-93bd-4016-8509-e44e8832d385 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:43.609912 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13849889-93bd-4016-8509-e44e8832d385 None admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:43.611450 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13849889-93bd-4016-8509-e44e8832d385 None admin] RBAC: Adding query filter params (name=service) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:43.611945 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13849889-93bd-4016-8509-e44e8832d385 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:43.614924 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13849889-93bd-4016-8509-e44e8832d385 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:43.636767 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 317/635] 10.222.0.55 () {56 vars in 1103 bytes} [Wed Aug 24 08:34:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:43.638757 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13849889-93bd-4016-8509-e44e8832d385 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:43.763197 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:43.774170 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867ef742-e425-4b57-87c7-aef7101ba19d None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:43.833593 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d63365-9c56-4cc1-827e-307896c4448c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:43.834343 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d63365-9c56-4cc1-827e-307896c4448c None admin] REQUEST_METHOD: `PUT` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:43.834701 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d63365-9c56-4cc1-827e-307896c4448c None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:43.835063 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d63365-9c56-4cc1-827e-307896c4448c None admin] PATH_INFO: `/v3/projects/420b18725f2342159646fb86438318fd/users/a0ebb5ccabef45378f0c1da69721e051/roles/bab90e02eebc473e9b911c8660ed9b76` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:43.836051 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:43.844269 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13849889-93bd-4016-8509-e44e8832d385 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:43.852936 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d63365-9c56-4cc1-827e-307896c4448c None admin] RBAC: Authorizing `identity:create_grant(project_id=420b18725f2342159646fb86438318fd, user_id=a0ebb5ccabef45378f0c1da69721e051, role_id=bab90e02eebc473e9b911c8660ed9b76)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:43.854524 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d63365-9c56-4cc1-827e-307896c4448c None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:43.884362 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c035694-4c59-4314-908e-7d351b025363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:43.885112 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c035694-4c59-4314-908e-7d351b025363 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:43.885426 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c035694-4c59-4314-908e-7d351b025363 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:43.885730 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c035694-4c59-4314-908e-7d351b025363 None admin] PATH_INFO: `/v3/domains` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:43.886439 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c035694-4c59-4314-908e-7d351b025363 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:43.886775 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c035694-4c59-4314-908e-7d351b025363 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:43.888803 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c035694-4c59-4314-908e-7d351b025363 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:43.901849 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 318/636] 10.222.0.55 () {56 vars in 1109 bytes} [Wed Aug 24 08:34:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:43.903237 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c035694-4c59-4314-908e-7d351b025363 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:43.936584 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 319/637] 10.222.0.55 () {58 vars in 1439 bytes} [Wed Aug 24 08:34:43 2022] PUT /identity/v3/projects/420b18725f2342159646fb86438318fd/users/a0ebb5ccabef45378f0c1da69721e051/roles/bab90e02eebc473e9b911c8660ed9b76 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:34:43.936584 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d63365-9c56-4cc1-827e-307896c4448c None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:44.050642 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c035694-4c59-4314-908e-7d351b025363 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:44.070793 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d63365-9c56-4cc1-827e-307896c4448c None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:44.099382 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffdb849-7432-4649-873f-f6a48279c968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:44.100027 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffdb849-7432-4649-873f-f6a48279c968 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:44.100321 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffdb849-7432-4649-873f-f6a48279c968 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:44.100605 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffdb849-7432-4649-873f-f6a48279c968 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:44.112148 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffdb849-7432-4649-873f-f6a48279c968 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:44.113678 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffdb849-7432-4649-873f-f6a48279c968 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:44.130313 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b7a601-3f16-4d9b-b3e7-ae6405c72e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:44.130876 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-1ffdb849-7432-4649-873f-f6a48279c968 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:34:44.130876 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:44.130876 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:44.130876 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:44.130876 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:44.130876 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:44.130876 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:44.130876 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:44.130876 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:44.130876 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:44.130876 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:44.130876 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:44.130876 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:34:44.130876 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:34:44.130876 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:34:44.130876 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:34:44.131864 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:44.131864 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:44.131864 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:44.131864 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:44.131864 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:44.131864 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:44.131864 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:44.131864 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:44.131864 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:44.131864 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:44.131864 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:44.131864 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:44.131864 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:44.131864 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:44.131864 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:44.131864 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:44.131864 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:34:44.133281 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:34:44.133281 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:34:44.133281 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:34:44.133281 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 319/638] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:34:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:44.133546 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b7a601-3f16-4d9b-b3e7-ae6405c72e28 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:44.133546 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b7a601-3f16-4d9b-b3e7-ae6405c72e28 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:44.133546 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b7a601-3f16-4d9b-b3e7-ae6405c72e28 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:44.133738 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffdb849-7432-4649-873f-f6a48279c968 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:44.146744 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:44.151558 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b7a601-3f16-4d9b-b3e7-ae6405c72e28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:34:44.157185 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-34b7a601-3f16-4d9b-b3e7-ae6405c72e28 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:34:44.157185 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:44.157185 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:44.157185 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:44.157185 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:44.157185 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:44.157185 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:44.157185 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:44.157185 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:44.157185 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:44.157185 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:44.157185 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:44.157185 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:34:44.157185 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:34:44.157185 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:34:44.157185 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:34:44.158995 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:34:44.158995 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:34:44.158995 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:34:44.158995 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:34:44.158995 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:44.158995 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:44.158995 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:44.158995 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:44.158995 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:44.158995 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:44.158995 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:44.158995 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:44.158995 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:44.158995 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:44.158995 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:44.158995 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:44.159913 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:44.159913 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:44.159913 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:44.159913 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:44.159913 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:34:44.159913 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:34:44.159913 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:34:44.159913 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:34:44.159913 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:34:44.159913 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:34:44.159913 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:34:44.159913 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:34:44.161159 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 320/639] 10.222.0.55 () {56 vars in 1102 bytes} [Wed Aug 24 08:34:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:44.162029 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556825d0-2f03-486b-9198-190a1d6e530d None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:44.162313 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556825d0-2f03-486b-9198-190a1d6e530d None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:44.162585 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556825d0-2f03-486b-9198-190a1d6e530d None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:44.165410 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 321/640] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:34:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:34:44.166086 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556825d0-2f03-486b-9198-190a1d6e530d None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:44.179090 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:44.268643 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffdb849-7432-4649-873f-f6a48279c968 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:44.293430 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556825d0-2f03-486b-9198-190a1d6e530d None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:44.310740 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986a086c-5845-4edb-96ce-79a3a069dd38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:44.312098 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986a086c-5845-4edb-96ce-79a3a069dd38 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:44.312384 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986a086c-5845-4edb-96ce-79a3a069dd38 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:44.312692 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986a086c-5845-4edb-96ce-79a3a069dd38 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:44.320357 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986a086c-5845-4edb-96ce-79a3a069dd38 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:44.321868 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986a086c-5845-4edb-96ce-79a3a069dd38 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:44.334497 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-986a086c-5845-4edb-96ce-79a3a069dd38 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:34:44.334497 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:44.334497 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:44.334497 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:44.334497 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:44.334497 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:44.334497 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:44.334497 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:44.334497 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:44.334497 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:44.334497 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:44.334497 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:44.334497 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:34:44.334497 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:34:44.334497 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:34:44.335534 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:34:44.335534 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:44.335534 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:44.335534 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:44.335534 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:44.335534 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:44.335534 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:44.335534 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:44.335534 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:44.335534 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:44.335534 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:44.335534 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:44.335534 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:44.335534 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:44.335534 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:44.335534 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:44.335534 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:44.336471 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:34:44.336471 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:34:44.336471 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:34:44.336471 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:34:44.336471 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:34:44.336471 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:34:44.336471 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:34:44.336471 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:34:44.336471 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 320/641] 10.222.0.55 () {56 vars in 1111 bytes} [Wed Aug 24 08:34:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:44.337013 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986a086c-5845-4edb-96ce-79a3a069dd38 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:44.339129 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eb8417-7f92-443a-9f6a-aabc9f68b1b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:44.340145 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eb8417-7f92-443a-9f6a-aabc9f68b1b8 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:44.340735 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eb8417-7f92-443a-9f6a-aabc9f68b1b8 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:44.340900 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eb8417-7f92-443a-9f6a-aabc9f68b1b8 None admin] PATH_INFO: `/v3/domains` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:44.341577 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8eb8417-7f92-443a-9f6a-aabc9f68b1b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:44.342010 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8eb8417-7f92-443a-9f6a-aabc9f68b1b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:44.343559 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8eb8417-7f92-443a-9f6a-aabc9f68b1b8 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:44.364220 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 322/642] 10.222.0.55 () {56 vars in 1109 bytes} [Wed Aug 24 08:34:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:44.365689 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfdaa22-aabd-404a-8dac-4f6513f73a0a None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:44.366027 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfdaa22-aabd-404a-8dac-4f6513f73a0a None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:44.366304 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfdaa22-aabd-404a-8dac-4f6513f73a0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:44.388809 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-cdfdaa22-aabd-404a-8dac-4f6513f73a0a None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:44.390128 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-cdfdaa22-aabd-404a-8dac-4f6513f73a0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:44.416986 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:44.454385 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986a086c-5845-4edb-96ce-79a3a069dd38 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:44.467589 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:44.480610 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 323/643] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:34:44.480610 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfdaa22-aabd-404a-8dac-4f6513f73a0a None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:44.504315 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9e21fb-f565-4c91-bb1e-b803b188a2e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:44.505062 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9e21fb-f565-4c91-bb1e-b803b188a2e2 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:44.505412 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9e21fb-f565-4c91-bb1e-b803b188a2e2 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:44.505786 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9e21fb-f565-4c91-bb1e-b803b188a2e2 None admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:44.506825 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9e21fb-f565-4c91-bb1e-b803b188a2e2 None admin] RBAC: Adding query filter params (name=service) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:44.507202 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9e21fb-f565-4c91-bb1e-b803b188a2e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:44.508869 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9e21fb-f565-4c91-bb1e-b803b188a2e2 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:44.522116 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 321/644] 10.222.0.55 () {56 vars in 1103 bytes} [Wed Aug 24 08:34:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:44.524056 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9e21fb-f565-4c91-bb1e-b803b188a2e2 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:44.632128 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfdaa22-aabd-404a-8dac-4f6513f73a0a None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:44.671749 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba575f4-69d5-4a36-ac24-b5c0b72d71fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:44.672528 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba575f4-69d5-4a36-ac24-b5c0b72d71fe None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:44.672945 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba575f4-69d5-4a36-ac24-b5c0b72d71fe None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:44.673360 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba575f4-69d5-4a36-ac24-b5c0b72d71fe None admin] PATH_INFO: `/v3/projects` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:44.674152 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba575f4-69d5-4a36-ac24-b5c0b72d71fe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:44.674570 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba575f4-69d5-4a36-ac24-b5c0b72d71fe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:44.676230 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba575f4-69d5-4a36-ac24-b5c0b72d71fe None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:44.677588 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9e21fb-f565-4c91-bb1e-b803b188a2e2 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:44.690725 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 324/645] 10.222.0.55 () {56 vars in 1148 bytes} [Wed Aug 24 08:34:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:44.691685 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba575f4-69d5-4a36-ac24-b5c0b72d71fe None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:44.718620 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:44.728990 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5d9401-654f-4149-b37f-863967ce6269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:44.730517 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5d9401-654f-4149-b37f-863967ce6269 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:44.731761 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5d9401-654f-4149-b37f-863967ce6269 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:44.732774 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5d9401-654f-4149-b37f-863967ce6269 None admin] PATH_INFO: `/v3/users/glance` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:44.754451 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5d9401-654f-4149-b37f-863967ce6269 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:44.760606 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5d9401-654f-4149-b37f-863967ce6269 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:44.773751 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:44.785550 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-8c5d9401-654f-4149-b37f-863967ce6269 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 08:34:44.785550 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:44.785550 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:44.785550 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:44.785550 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:44.785550 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:44.785550 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:44.785550 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:44.785550 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:44.785550 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:44.785550 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:44.785550 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:44.785550 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:34:44.785550 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:34:44.785550 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:34:44.785550 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:34:44.787345 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:44.787345 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:44.787345 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:34:44.787345 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:34:44.787345 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:34:44.787345 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:34:44.787345 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:44.787345 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:44.787345 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:44.787345 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:44.787345 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:44.787345 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:44.787345 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:44.787345 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:44.787345 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:44.787345 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:44.787345 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:44.787345 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:44.789148 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:44.789148 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:44.789148 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:34:44.789148 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:34:44.789148 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:34:44.789148 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:34:44.789148 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:34:44.789148 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:34:44.789148 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 08:34:44.789148 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:34:44.789148 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 322/646] 10.222.0.55 () {56 vars in 1099 bytes} [Wed Aug 24 08:34:44 2022] GET /identity/v3/users/glance => generated 84 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:44.789937 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5d9401-654f-4149-b37f-863967ce6269 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:44.863479 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba575f4-69d5-4a36-ac24-b5c0b72d71fe None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:44.923431 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5d9401-654f-4149-b37f-863967ce6269 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:44.926027 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a742c6-4a60-4d8c-8d60-aaefc3a25878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:44.927014 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a742c6-4a60-4d8c-8d60-aaefc3a25878 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:44.927503 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a742c6-4a60-4d8c-8d60-aaefc3a25878 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:44.927976 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a742c6-4a60-4d8c-8d60-aaefc3a25878 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:44.947463 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a742c6-4a60-4d8c-8d60-aaefc3a25878 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:44.950255 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a742c6-4a60-4d8c-8d60-aaefc3a25878 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:44.968311 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f221e1-67ab-4d60-9773-eaaf8d745fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:44.969059 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f221e1-67ab-4d60-9773-eaaf8d745fbd None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:44.969338 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f221e1-67ab-4d60-9773-eaaf8d745fbd None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:44.969610 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f221e1-67ab-4d60-9773-eaaf8d745fbd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:44.970338 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f221e1-67ab-4d60-9773-eaaf8d745fbd None admin] RBAC: Adding query filter params (role.id=bab90e02eebc473e9b911c8660ed9b76, scope.project.id=420b18725f2342159646fb86438318fd, user.id=f17da357e555420799ee681bffbcc268) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:44.970648 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f221e1-67ab-4d60-9773-eaaf8d745fbd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:44.973351 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f221e1-67ab-4d60-9773-eaaf8d745fbd None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:44.978564 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-f1a742c6-4a60-4d8c-8d60-aaefc3a25878 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:34:44.978564 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:44.978564 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:44.978564 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:44.978564 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:44.978564 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:44.978564 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:44.978564 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:44.978564 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:44.978564 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:44.978564 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:44.978564 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:44.978564 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:34:44.978564 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:34:44.978564 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:34:44.978564 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:34:44.989135 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:44.989135 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:44.989135 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:44.989135 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:44.989135 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:44.989135 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:44.989135 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:44.989135 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:44.989135 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:44.989135 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:44.989135 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:44.989135 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:44.989135 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:44.989135 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:44.989135 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:44.989135 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:44.989135 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:34:44.990811 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:34:44.990811 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:34:44.990811 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:34:44.990811 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 325/647] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:34:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:44.990811 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a742c6-4a60-4d8c-8d60-aaefc3a25878 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:44.991342 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 323/648] 10.222.0.55 () {56 vars in 1374 bytes} [Wed Aug 24 08:34:44 2022] GET /identity/v3/role_assignments?user.id=f17da357e555420799ee681bffbcc268&scope.project.id=420b18725f2342159646fb86438318fd&role.id=bab90e02eebc473e9b911c8660ed9b76 => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:44.991342 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f221e1-67ab-4d60-9773-eaaf8d745fbd None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:45.120034 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:45.217843 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f221e1-67ab-4d60-9773-eaaf8d745fbd None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:45.227634 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a742c6-4a60-4d8c-8d60-aaefc3a25878 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:45.254469 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:45.289695 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20dfc94-cf0e-4c9e-8f24-6405383e76fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:45.290751 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20dfc94-cf0e-4c9e-8f24-6405383e76fa None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:45.291239 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20dfc94-cf0e-4c9e-8f24-6405383e76fa None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:45.291723 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20dfc94-cf0e-4c9e-8f24-6405383e76fa None admin] PATH_INFO: `/v3/users` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:45.293119 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20dfc94-cf0e-4c9e-8f24-6405383e76fa None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:45.293983 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20dfc94-cf0e-4c9e-8f24-6405383e76fa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:45.298473 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20dfc94-cf0e-4c9e-8f24-6405383e76fa None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:45.308251 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f6dae1-2582-4689-94f3-dae28ac51159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:45.309676 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f6dae1-2582-4689-94f3-dae28ac51159 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:45.310153 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f6dae1-2582-4689-94f3-dae28ac51159 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:45.310616 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f6dae1-2582-4689-94f3-dae28ac51159 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:45.330891 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f6dae1-2582-4689-94f3-dae28ac51159 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:34:45.333524 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 324/649] 10.222.0.55 () {56 vars in 1137 bytes} [Wed Aug 24 08:34:44 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:45.335109 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20dfc94-cf0e-4c9e-8f24-6405383e76fa None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:45.343517 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-c7f6dae1-2582-4689-94f3-dae28ac51159 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:34:45.343517 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:45.343517 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:45.343517 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:45.343517 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:45.343517 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:45.343517 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:45.343517 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:45.343517 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:45.343517 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:45.343517 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:45.343517 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:45.343517 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:34:45.343517 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:34:45.343517 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:34:45.343517 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:34:45.345415 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:34:45.345415 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:34:45.345415 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:34:45.345415 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:34:45.345415 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:45.345415 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:45.345415 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:45.345415 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:45.345415 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:45.345415 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:45.345415 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:45.345415 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:45.345415 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:45.345415 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:45.345415 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:45.345415 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:45.347073 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:45.347073 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:45.347073 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:45.347073 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:45.347073 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:34:45.347073 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:34:45.347073 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:34:45.347073 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:34:45.347073 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:34:45.347073 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:34:45.347073 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:34:45.347073 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:34:45.347073 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 326/650] 10.222.0.55 () {56 vars in 1102 bytes} [Wed Aug 24 08:34:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 365 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:45.348446 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f6dae1-2582-4689-94f3-dae28ac51159 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:45.539576 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f6dae1-2582-4689-94f3-dae28ac51159 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:45.566708 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20dfc94-cf0e-4c9e-8f24-6405383e76fa None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:45.593153 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:45.621244 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9299cd7-c2d9-4696-bf28-47ef7f5a641c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:45.622962 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9299cd7-c2d9-4696-bf28-47ef7f5a641c None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:45.623734 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9299cd7-c2d9-4696-bf28-47ef7f5a641c None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:45.625278 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9299cd7-c2d9-4696-bf28-47ef7f5a641c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:45.634614 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c3c50b-6ea1-40b3-863b-3c8ca951c95d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:45.634614 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c3c50b-6ea1-40b3-863b-3c8ca951c95d None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:45.635062 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c3c50b-6ea1-40b3-863b-3c8ca951c95d None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:45.635321 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c3c50b-6ea1-40b3-863b-3c8ca951c95d None admin] PATH_INFO: `/v3/domains` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:45.636585 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c3c50b-6ea1-40b3-863b-3c8ca951c95d None admin] RBAC: Adding query filter params (name=Default) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:45.637732 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c3c50b-6ea1-40b3-863b-3c8ca951c95d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:45.640285 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c3c50b-6ea1-40b3-863b-3c8ca951c95d None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:45.663294 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 325/651] 10.222.0.55 () {56 vars in 1109 bytes} [Wed Aug 24 08:34:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:45.665412 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c3c50b-6ea1-40b3-863b-3c8ca951c95d None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:45.698430 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:45.771830 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9299cd7-c2d9-4696-bf28-47ef7f5a641c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:45.775730 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9299cd7-c2d9-4696-bf28-47ef7f5a641c None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:45.867740 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c3c50b-6ea1-40b3-863b-3c8ca951c95d None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:45.929231 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d026e2c8-5b4e-4f8f-a960-343d9bb83ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:45.929231 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d026e2c8-5b4e-4f8f-a960-343d9bb83ead None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:45.929666 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d026e2c8-5b4e-4f8f-a960-343d9bb83ead None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:45.930044 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d026e2c8-5b4e-4f8f-a960-343d9bb83ead None admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:45.931203 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d026e2c8-5b4e-4f8f-a960-343d9bb83ead None admin] RBAC: Adding query filter params (name=service) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:45.931727 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d026e2c8-5b4e-4f8f-a960-343d9bb83ead None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:45.934788 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d026e2c8-5b4e-4f8f-a960-343d9bb83ead None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:45.952959 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 326/652] 10.222.0.55 () {56 vars in 1103 bytes} [Wed Aug 24 08:34:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:45.952959 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d026e2c8-5b4e-4f8f-a960-343d9bb83ead None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:46.009995 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 327/653] 10.222.0.55 () {58 vars in 1282 bytes} [Wed Aug 24 08:34:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 663 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:34:46.013018 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9299cd7-c2d9-4696-bf28-47ef7f5a641c None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:46.023234 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:46.141277 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d026e2c8-5b4e-4f8f-a960-343d9bb83ead None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:46.144465 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:46.200588 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca40834-9552-4f53-a87e-0b90a57cf964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:46.201269 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca40834-9552-4f53-a87e-0b90a57cf964 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:46.201553 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca40834-9552-4f53-a87e-0b90a57cf964 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:46.201831 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca40834-9552-4f53-a87e-0b90a57cf964 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:46.211472 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca40834-9552-4f53-a87e-0b90a57cf964 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:46.213039 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9299cd7-c2d9-4696-bf28-47ef7f5a641c None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:46.213231 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca40834-9552-4f53-a87e-0b90a57cf964 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:46.226632 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-5ca40834-9552-4f53-a87e-0b90a57cf964 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 08:34:46.226632 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:46.226632 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:46.226632 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:46.226632 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:46.226632 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:46.226632 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:46.226632 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:46.226632 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:46.226632 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:46.226632 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:46.226632 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:46.226632 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:34:46.226632 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:34:46.226632 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:34:46.226632 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:34:46.228094 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:46.228094 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:46.228094 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:34:46.228094 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:34:46.228094 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:34:46.228094 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:34:46.228094 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:46.228094 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:46.228094 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:46.228094 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:46.228094 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:46.228094 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:46.228094 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:46.228094 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:46.228094 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:46.228094 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:46.228094 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:46.228094 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:46.230148 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:46.230148 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:46.230148 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:34:46.230148 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:34:46.230148 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:34:46.230148 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:34:46.230148 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:34:46.230148 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:34:46.230148 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 08:34:46.230148 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:34:46.230148 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 327/654] 10.222.0.55 () {56 vars in 1102 bytes} [Wed Aug 24 08:34:45 2022] GET /identity/v3/users/neutron => generated 85 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:46.230148 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca40834-9552-4f53-a87e-0b90a57cf964 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:46.266502 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde67464-120b-42f4-9ea4-eebd449a6778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:46.267144 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde67464-120b-42f4-9ea4-eebd449a6778 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:46.267420 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde67464-120b-42f4-9ea4-eebd449a6778 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:46.267683 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde67464-120b-42f4-9ea4-eebd449a6778 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:46.279952 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde67464-120b-42f4-9ea4-eebd449a6778 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:46.281450 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde67464-120b-42f4-9ea4-eebd449a6778 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:46.299541 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-fde67464-120b-42f4-9ea4-eebd449a6778 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:34:46.299541 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:46.299541 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:46.299541 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:46.299541 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:46.299541 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:46.299541 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:46.299541 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:46.299541 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:46.299541 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:46.299541 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:46.299541 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:46.299541 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:34:46.299541 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:34:46.299541 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:34:46.299541 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:34:46.300600 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:46.300600 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:46.300600 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:46.300600 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:46.300600 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:46.300600 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:46.300600 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:46.300600 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:46.300600 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:46.300600 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:46.300600 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:46.300600 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:46.300600 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:46.300600 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:46.300600 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:46.300600 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:46.300600 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:34:46.302402 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:34:46.302402 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:34:46.302402 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:34:46.305672 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 328/655] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:34:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:46.306967 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19fe99b-e2f3-45ce-860c-2e030fbca886 None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:46.307244 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19fe99b-e2f3-45ce-860c-2e030fbca886 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:46.307529 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19fe99b-e2f3-45ce-860c-2e030fbca886 None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:46.309790 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 329/656] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:34:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:34:46.310531 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19fe99b-e2f3-45ce-860c-2e030fbca886 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:46.378676 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:46.392237 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca40834-9552-4f53-a87e-0b90a57cf964 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:46.455416 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8605e536-8a97-406b-a38a-d42091d65a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:46.456076 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8605e536-8a97-406b-a38a-d42091d65a92 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:46.456356 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8605e536-8a97-406b-a38a-d42091d65a92 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:46.456672 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8605e536-8a97-406b-a38a-d42091d65a92 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:46.469668 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8605e536-8a97-406b-a38a-d42091d65a92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:46.471170 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8605e536-8a97-406b-a38a-d42091d65a92 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:46.471633 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19fe99b-e2f3-45ce-860c-2e030fbca886 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:46.487902 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-8605e536-8a97-406b-a38a-d42091d65a92 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:34:46.487902 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:46.487902 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:46.487902 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:46.487902 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:46.487902 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:46.487902 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:46.487902 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:46.487902 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:46.487902 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:46.487902 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:46.487902 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:46.487902 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:34:46.487902 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:34:46.487902 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:34:46.487902 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:34:46.493105 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:46.493105 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:46.493105 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:46.493105 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:46.493105 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:46.493105 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:46.493105 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:46.493105 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:46.493105 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:46.493105 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:46.493105 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:46.493105 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:46.493105 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:46.493105 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:46.493105 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:46.493105 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:46.493105 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:34:46.495619 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:34:46.495619 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:34:46.495619 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:34:46.495619 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 328/657] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:34:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:46.495619 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8605e536-8a97-406b-a38a-d42091d65a92 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:46.511907 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:46.529410 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3e494d-7f59-4c3f-87ff-f7e0a7972c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:46.530053 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3e494d-7f59-4c3f-87ff-f7e0a7972c76 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:46.530310 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3e494d-7f59-4c3f-87ff-f7e0a7972c76 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:46.530600 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3e494d-7f59-4c3f-87ff-f7e0a7972c76 None admin] PATH_INFO: `/v3/users` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:46.531317 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3e494d-7f59-4c3f-87ff-f7e0a7972c76 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:46.531613 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3e494d-7f59-4c3f-87ff-f7e0a7972c76 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:46.533130 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3e494d-7f59-4c3f-87ff-f7e0a7972c76 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:46.560511 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 330/658] 10.222.0.55 () {56 vars in 1139 bytes} [Wed Aug 24 08:34:46 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:46.560511 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50780e84-19a1-4e34-bce1-f4ef1e156789 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:46.560511 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50780e84-19a1-4e34-bce1-f4ef1e156789 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:46.560879 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50780e84-19a1-4e34-bce1-f4ef1e156789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:46.591404 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-50780e84-19a1-4e34-bce1-f4ef1e156789 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:46.593447 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-50780e84-19a1-4e34-bce1-f4ef1e156789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:46.625918 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8605e536-8a97-406b-a38a-d42091d65a92 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:46.665523 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994bf6dd-26fb-47a9-a00d-c6993e9a3d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:46.666063 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994bf6dd-26fb-47a9-a00d-c6993e9a3d85 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:46.666340 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994bf6dd-26fb-47a9-a00d-c6993e9a3d85 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:46.666609 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994bf6dd-26fb-47a9-a00d-c6993e9a3d85 None admin] PATH_INFO: `/v3/domains` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:46.667254 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994bf6dd-26fb-47a9-a00d-c6993e9a3d85 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:46.667552 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994bf6dd-26fb-47a9-a00d-c6993e9a3d85 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:46.670162 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994bf6dd-26fb-47a9-a00d-c6993e9a3d85 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:46.689045 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 329/659] 10.222.0.55 () {56 vars in 1109 bytes} [Wed Aug 24 08:34:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:46.689045 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994bf6dd-26fb-47a9-a00d-c6993e9a3d85 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:46.697605 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:46.728271 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 331/660] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:34:46.728271 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50780e84-19a1-4e34-bce1-f4ef1e156789 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:46.818703 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994bf6dd-26fb-47a9-a00d-c6993e9a3d85 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:46.858023 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54e8cf3-f6cc-4c17-ac0a-c8081d5f456b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:46.858933 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54e8cf3-f6cc-4c17-ac0a-c8081d5f456b None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:46.859285 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54e8cf3-f6cc-4c17-ac0a-c8081d5f456b None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:46.859632 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54e8cf3-f6cc-4c17-ac0a-c8081d5f456b None admin] PATH_INFO: `/v3/domains` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:46.860353 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54e8cf3-f6cc-4c17-ac0a-c8081d5f456b None admin] RBAC: Adding query filter params (name=Default) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:46.860779 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54e8cf3-f6cc-4c17-ac0a-c8081d5f456b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:46.862264 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54e8cf3-f6cc-4c17-ac0a-c8081d5f456b None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:46.875403 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 330/661] 10.222.0.55 () {56 vars in 1109 bytes} [Wed Aug 24 08:34:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:46.876377 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54e8cf3-f6cc-4c17-ac0a-c8081d5f456b None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:46.926967 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50780e84-19a1-4e34-bce1-f4ef1e156789 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:46.950924 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:46.953067 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:46.991764 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54e8cf3-f6cc-4c17-ac0a-c8081d5f456b None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:47.000806 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c0edb7-758f-4981-a20b-32234028a546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:47.010269 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c0edb7-758f-4981-a20b-32234028a546 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:47.010269 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c0edb7-758f-4981-a20b-32234028a546 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:47.010269 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c0edb7-758f-4981-a20b-32234028a546 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:47.036460 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73616fb8-3f55-4081-a618-6709cc54d869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:47.037334 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73616fb8-3f55-4081-a618-6709cc54d869 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:47.037808 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73616fb8-3f55-4081-a618-6709cc54d869 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:47.038280 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73616fb8-3f55-4081-a618-6709cc54d869 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:47.049651 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73616fb8-3f55-4081-a618-6709cc54d869 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:47.052334 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73616fb8-3f55-4081-a618-6709cc54d869 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:47.073960 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-73616fb8-3f55-4081-a618-6709cc54d869 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 08:34:47.073960 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:47.073960 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:47.073960 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:47.073960 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:47.073960 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:47.073960 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:47.073960 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:47.073960 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:47.073960 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:47.073960 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:47.073960 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:47.073960 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:34:47.073960 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:34:47.073960 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:34:47.073960 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:34:47.075546 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:47.075546 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:47.075546 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:34:47.075546 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:34:47.075546 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:34:47.075546 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:34:47.075546 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:47.075546 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:47.075546 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:47.075546 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:47.075546 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:47.075546 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:47.075546 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:47.075546 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:47.075546 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:47.075546 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:47.075546 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:47.075546 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:47.076504 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:47.076504 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:47.076504 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:34:47.076504 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:34:47.076504 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:34:47.076504 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:34:47.076504 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:34:47.076504 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:34:47.076504 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 08:34:47.076504 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:34:47.077079 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 331/662] 10.222.0.55 () {56 vars in 1099 bytes} [Wed Aug 24 08:34:46 2022] GET /identity/v3/users/cinder => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:47.078325 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73616fb8-3f55-4081-a618-6709cc54d869 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:47.116007 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c0edb7-758f-4981-a20b-32234028a546 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:47.120525 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c0edb7-758f-4981-a20b-32234028a546 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:47.217203 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73616fb8-3f55-4081-a618-6709cc54d869 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:47.249334 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:47.264587 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 332/663] 10.222.0.55 () {58 vars in 1282 bytes} [Wed Aug 24 08:34:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 538 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:34:47.264587 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c0edb7-758f-4981-a20b-32234028a546 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:47.268385 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc2f7b7-0322-414f-b3c8-75305cdbb146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:47.268385 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc2f7b7-0322-414f-b3c8-75305cdbb146 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:47.268385 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc2f7b7-0322-414f-b3c8-75305cdbb146 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:47.268385 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc2f7b7-0322-414f-b3c8-75305cdbb146 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:47.275468 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc2f7b7-0322-414f-b3c8-75305cdbb146 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:47.276290 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:47.277456 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc2f7b7-0322-414f-b3c8-75305cdbb146 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:47.293692 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-dcc2f7b7-0322-414f-b3c8-75305cdbb146 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:34:47.293692 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:47.293692 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:47.293692 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:47.293692 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:47.293692 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:47.293692 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:47.293692 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:47.293692 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:47.293692 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:47.293692 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:47.293692 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:47.293692 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:34:47.293692 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:34:47.293692 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:34:47.295818 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:34:47.295818 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:47.295818 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:47.295818 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:47.295818 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:47.295818 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:47.295818 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:47.295818 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:47.295818 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:47.295818 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:47.295818 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:47.295818 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:47.295818 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:47.295818 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:47.295818 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:47.295818 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:47.295818 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:47.303647 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:34:47.303647 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:34:47.303647 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:34:47.303647 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:34:47.303647 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:34:47.303647 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:34:47.303647 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:34:47.303647 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:34:47.304961 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 332/664] 10.222.0.55 () {56 vars in 1111 bytes} [Wed Aug 24 08:34:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:47.306844 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc2f7b7-0322-414f-b3c8-75305cdbb146 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:47.380442 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c0edb7-758f-4981-a20b-32234028a546 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:47.417960 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6964d569-20a5-4d27-baf7-c607f7d8794c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:47.418644 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6964d569-20a5-4d27-baf7-c607f7d8794c None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:47.418924 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6964d569-20a5-4d27-baf7-c607f7d8794c None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:47.419197 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6964d569-20a5-4d27-baf7-c607f7d8794c None admin] PATH_INFO: `/v3/roles/service` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:47.430288 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6964d569-20a5-4d27-baf7-c607f7d8794c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:34:47.435544 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-6964d569-20a5-4d27-baf7-c607f7d8794c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:34:47.435544 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:47.435544 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:47.435544 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:47.435544 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:47.435544 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:47.435544 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:47.435544 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:47.435544 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:47.435544 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:47.435544 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:47.435544 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:47.435544 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:34:47.435544 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:34:47.435544 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:34:47.435544 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:34:47.436558 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:34:47.436558 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:34:47.436558 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:34:47.436558 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:34:47.436558 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:47.436558 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:47.436558 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:47.436558 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:47.436558 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:47.436558 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:47.436558 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:47.436558 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:47.436558 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:47.436558 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:47.436558 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:47.436558 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:47.437446 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:47.437446 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:47.437446 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:47.437446 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:47.437446 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:34:47.437446 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:34:47.437446 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:34:47.437446 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:34:47.437446 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:34:47.437446 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:34:47.437446 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:34:47.437446 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:34:47.437446 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 333/665] 10.222.0.55 () {56 vars in 1102 bytes} [Wed Aug 24 08:34:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:47.438131 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6964d569-20a5-4d27-baf7-c607f7d8794c None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:47.500407 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc2f7b7-0322-414f-b3c8-75305cdbb146 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:47.517168 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:47.554417 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b00b38-2f16-4dec-9c42-2b0c7871767f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:47.555093 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b00b38-2f16-4dec-9c42-2b0c7871767f None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:47.555889 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b00b38-2f16-4dec-9c42-2b0c7871767f None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:47.556555 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b00b38-2f16-4dec-9c42-2b0c7871767f None admin] PATH_INFO: `/v3/users` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:47.557466 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b00b38-2f16-4dec-9c42-2b0c7871767f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:47.557830 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b00b38-2f16-4dec-9c42-2b0c7871767f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:47.559969 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b00b38-2f16-4dec-9c42-2b0c7871767f None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:47.568621 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6964d569-20a5-4d27-baf7-c607f7d8794c None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:47.584639 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 333/666] 10.222.0.55 () {56 vars in 1137 bytes} [Wed Aug 24 08:34:47 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:47.584639 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b00b38-2f16-4dec-9c42-2b0c7871767f None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:47.633927 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:47.640433 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aafeb6-bbb6-4279-b952-866f9b4e968a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:47.641584 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aafeb6-bbb6-4279-b952-866f9b4e968a None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:47.642077 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aafeb6-bbb6-4279-b952-866f9b4e968a None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:47.642581 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aafeb6-bbb6-4279-b952-866f9b4e968a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:47.662662 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9aafeb6-bbb6-4279-b952-866f9b4e968a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:47.665433 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9aafeb6-bbb6-4279-b952-866f9b4e968a None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:47.692199 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b00b38-2f16-4dec-9c42-2b0c7871767f None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:47.696309 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-a9aafeb6-bbb6-4279-b952-866f9b4e968a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:34:47.696309 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:47.696309 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:47.696309 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:47.696309 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:47.696309 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:47.696309 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:47.696309 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:47.696309 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:47.696309 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:47.696309 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:47.696309 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:47.696309 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:34:47.696309 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:34:47.696309 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:34:47.696309 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:34:47.697324 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:47.697324 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:47.697324 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:47.697324 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:47.697324 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:47.697324 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:47.697324 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:47.697324 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:47.697324 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:47.697324 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:47.697324 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:47.697324 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:47.697324 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:47.697324 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:47.697324 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:47.697324 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:47.697324 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:34:47.698220 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:34:47.698220 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:34:47.698220 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:34:47.700889 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 334/667] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:34:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:47.701616 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aafeb6-bbb6-4279-b952-866f9b4e968a None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:47.738418 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718dd17b-5b4c-4cc2-ac4e-908e86a41134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:47.739243 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718dd17b-5b4c-4cc2-ac4e-908e86a41134 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:47.739519 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718dd17b-5b4c-4cc2-ac4e-908e86a41134 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:47.739796 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718dd17b-5b4c-4cc2-ac4e-908e86a41134 None admin] PATH_INFO: `/v3/projects` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:47.740501 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718dd17b-5b4c-4cc2-ac4e-908e86a41134 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:47.740839 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718dd17b-5b4c-4cc2-ac4e-908e86a41134 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:47.742284 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718dd17b-5b4c-4cc2-ac4e-908e86a41134 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:47.756778 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 334/668] 10.222.0.55 () {56 vars in 1148 bytes} [Wed Aug 24 08:34:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:47.757579 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718dd17b-5b4c-4cc2-ac4e-908e86a41134 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:47.830194 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aafeb6-bbb6-4279-b952-866f9b4e968a None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:47.853654 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:47.869121 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0ee344-5003-4f4d-82db-47e42f1577e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:47.869741 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0ee344-5003-4f4d-82db-47e42f1577e3 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:47.870018 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0ee344-5003-4f4d-82db-47e42f1577e3 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:47.870286 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0ee344-5003-4f4d-82db-47e42f1577e3 None admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:47.870975 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0ee344-5003-4f4d-82db-47e42f1577e3 None admin] RBAC: Adding query filter params (name=service) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:47.871252 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0ee344-5003-4f4d-82db-47e42f1577e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:47.873224 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0ee344-5003-4f4d-82db-47e42f1577e3 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:47.886847 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 335/669] 10.222.0.55 () {56 vars in 1103 bytes} [Wed Aug 24 08:34:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:47.886847 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0ee344-5003-4f4d-82db-47e42f1577e3 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:47.912185 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718dd17b-5b4c-4cc2-ac4e-908e86a41134 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:47.915852 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:47.966986 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76b62f9-1cd9-44d7-a206-2e5051cc7978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:47.967948 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76b62f9-1cd9-44d7-a206-2e5051cc7978 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:47.968301 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76b62f9-1cd9-44d7-a206-2e5051cc7978 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:47.968737 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76b62f9-1cd9-44d7-a206-2e5051cc7978 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:48.048432 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0ee344-5003-4f4d-82db-47e42f1577e3 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:48.057929 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76b62f9-1cd9-44d7-a206-2e5051cc7978 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:48.059499 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76b62f9-1cd9-44d7-a206-2e5051cc7978 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:48.110204 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef51f87-b9fa-4b48-8047-745c5b419777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:48.111218 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef51f87-b9fa-4b48-8047-745c5b419777 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:48.111714 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef51f87-b9fa-4b48-8047-745c5b419777 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:48.112187 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef51f87-b9fa-4b48-8047-745c5b419777 None admin] PATH_INFO: `/v3/domains` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:48.113334 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef51f87-b9fa-4b48-8047-745c5b419777 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:48.113904 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef51f87-b9fa-4b48-8047-745c5b419777 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:48.116308 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef51f87-b9fa-4b48-8047-745c5b419777 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:48.142766 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 336/670] 10.222.0.55 () {56 vars in 1109 bytes} [Wed Aug 24 08:34:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:48.142766 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef51f87-b9fa-4b48-8047-745c5b419777 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:48.170976 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 335/671] 10.222.0.55 () {58 vars in 1282 bytes} [Wed Aug 24 08:34:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:34:48.171847 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76b62f9-1cd9-44d7-a206-2e5051cc7978 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:48.175076 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:48.227615 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:48.299126 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76b62f9-1cd9-44d7-a206-2e5051cc7978 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:48.300944 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef51f87-b9fa-4b48-8047-745c5b419777 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:48.337580 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123056ad-327d-444d-a69a-6b6ff0f6f37c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:48.338218 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123056ad-327d-444d-a69a-6b6ff0f6f37c None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:48.338498 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123056ad-327d-444d-a69a-6b6ff0f6f37c None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:48.338768 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123056ad-327d-444d-a69a-6b6ff0f6f37c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:48.345479 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acccdc0e-7783-48c1-8b51-16a9ea4bd2c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:48.346538 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acccdc0e-7783-48c1-8b51-16a9ea4bd2c2 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:48.346948 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acccdc0e-7783-48c1-8b51-16a9ea4bd2c2 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:48.347346 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acccdc0e-7783-48c1-8b51-16a9ea4bd2c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:48.348592 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acccdc0e-7783-48c1-8b51-16a9ea4bd2c2 None admin] RBAC: Adding query filter params (role.id=bab90e02eebc473e9b911c8660ed9b76, scope.project.id=420b18725f2342159646fb86438318fd, user.id=5512043ca9a7421b821741ca88d50b07) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:48.349052 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acccdc0e-7783-48c1-8b51-16a9ea4bd2c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:48.350175 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123056ad-327d-444d-a69a-6b6ff0f6f37c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:48.350839 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acccdc0e-7783-48c1-8b51-16a9ea4bd2c2 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:48.351608 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123056ad-327d-444d-a69a-6b6ff0f6f37c None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:48.368573 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 337/672] 10.222.0.55 () {56 vars in 1374 bytes} [Wed Aug 24 08:34:48 2022] GET /identity/v3/role_assignments?user.id=5512043ca9a7421b821741ca88d50b07&scope.project.id=420b18725f2342159646fb86438318fd&role.id=bab90e02eebc473e9b911c8660ed9b76 => generated 608 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:48.368573 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acccdc0e-7783-48c1-8b51-16a9ea4bd2c2 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:48.384446 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-123056ad-327d-444d-a69a-6b6ff0f6f37c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:34:48.384446 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:48.384446 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:48.384446 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:48.384446 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:48.384446 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:48.384446 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:48.384446 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:48.384446 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:48.384446 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:48.384446 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:48.384446 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:48.384446 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:34:48.384446 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:34:48.384446 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:34:48.384446 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:34:48.385670 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:48.385670 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:48.385670 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:48.385670 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:48.385670 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:48.385670 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:48.385670 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:48.385670 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:48.385670 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:48.385670 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:48.385670 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:48.385670 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:48.385670 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:48.385670 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:48.385670 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:48.385670 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:48.385670 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:34:48.386749 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:34:48.386749 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:34:48.386749 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:34:48.387334 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 336/673] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:34:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:48.388836 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123056ad-327d-444d-a69a-6b6ff0f6f37c None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:48.515721 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:48.541812 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acccdc0e-7783-48c1-8b51-16a9ea4bd2c2 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:48.556241 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:48.598776 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58c559d-b38e-4ab8-8b20-297eeec988c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:48.599424 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58c559d-b38e-4ab8-8b20-297eeec988c9 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:48.599689 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58c559d-b38e-4ab8-8b20-297eeec988c9 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:48.599959 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58c559d-b38e-4ab8-8b20-297eeec988c9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:48.600603 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123056ad-327d-444d-a69a-6b6ff0f6f37c None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:48.614031 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58c559d-b38e-4ab8-8b20-297eeec988c9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:48.616872 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58c559d-b38e-4ab8-8b20-297eeec988c9 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:48.639944 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-b58c559d-b38e-4ab8-8b20-297eeec988c9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:34:48.639944 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:48.639944 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:48.639944 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:48.639944 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:48.639944 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:48.639944 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:48.639944 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:48.639944 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:48.639944 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:48.639944 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:48.639944 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:48.639944 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:34:48.639944 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:34:48.639944 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:34:48.640963 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:34:48.640963 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:48.640963 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:48.640963 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:48.640963 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:48.640963 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:48.640963 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:48.640963 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:48.640963 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:48.640963 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:48.640963 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:48.640963 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:48.640963 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:48.640963 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:48.640963 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:48.640963 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:48.640963 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:48.641864 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:34:48.641864 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:34:48.641864 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:34:48.641864 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:34:48.641864 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:34:48.641864 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:34:48.641864 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:34:48.641864 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:34:48.642289 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 338/674] 10.222.0.55 () {56 vars in 1111 bytes} [Wed Aug 24 08:34:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:48.643309 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58c559d-b38e-4ab8-8b20-297eeec988c9 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:48.655748 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76268717-e53c-4859-a65d-c83d59c44900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:48.656870 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76268717-e53c-4859-a65d-c83d59c44900 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:48.657364 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76268717-e53c-4859-a65d-c83d59c44900 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:48.657842 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76268717-e53c-4859-a65d-c83d59c44900 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:48.678259 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76268717-e53c-4859-a65d-c83d59c44900 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:48.681005 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76268717-e53c-4859-a65d-c83d59c44900 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:48.709195 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-76268717-e53c-4859-a65d-c83d59c44900 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:34:48.709195 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:48.709195 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:48.709195 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:48.709195 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:48.709195 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:48.709195 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:48.709195 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:48.709195 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:48.709195 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:48.709195 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:48.709195 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:48.709195 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:34:48.709195 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:34:48.709195 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:34:48.709195 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:34:48.711005 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:48.711005 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:48.711005 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:48.711005 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:48.711005 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:48.711005 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:48.711005 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:48.711005 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:48.711005 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:48.711005 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:48.711005 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:48.711005 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:48.711005 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:48.711005 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:48.711005 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:48.711005 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:48.711005 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:34:48.713437 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:34:48.713437 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:34:48.713437 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:34:48.713437 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 337/675] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:34:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:48.713437 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b8a7f0-98a9-44f7-9ea5-8ccea403ba33 None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:48.713979 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b8a7f0-98a9-44f7-9ea5-8ccea403ba33 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:48.714587 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b8a7f0-98a9-44f7-9ea5-8ccea403ba33 None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:48.720613 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 338/676] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:34:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:34:48.720613 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b8a7f0-98a9-44f7-9ea5-8ccea403ba33 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:48.777001 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58c559d-b38e-4ab8-8b20-297eeec988c9 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:48.820261 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98db4ad8-98f9-4753-82e3-3a3fc367fca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:48.820800 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98db4ad8-98f9-4753-82e3-3a3fc367fca4 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:48.821078 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98db4ad8-98f9-4753-82e3-3a3fc367fca4 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:48.821349 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98db4ad8-98f9-4753-82e3-3a3fc367fca4 None admin] PATH_INFO: `/v3/domains` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:48.821997 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98db4ad8-98f9-4753-82e3-3a3fc367fca4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:48.822304 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98db4ad8-98f9-4753-82e3-3a3fc367fca4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:48.823721 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98db4ad8-98f9-4753-82e3-3a3fc367fca4 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:48.855172 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 339/677] 10.222.0.55 () {56 vars in 1109 bytes} [Wed Aug 24 08:34:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:48.855172 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98db4ad8-98f9-4753-82e3-3a3fc367fca4 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:48.882065 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b8a7f0-98a9-44f7-9ea5-8ccea403ba33 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:48.893991 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:48.910884 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:48.929710 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ce629b-c2d1-4336-93de-9d938284eac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:48.930706 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ce629b-c2d1-4336-93de-9d938284eac6 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:48.931180 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ce629b-c2d1-4336-93de-9d938284eac6 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:48.931649 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ce629b-c2d1-4336-93de-9d938284eac6 None admin] PATH_INFO: `/v3/projects` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:48.933170 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ce629b-c2d1-4336-93de-9d938284eac6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:48.933653 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ce629b-c2d1-4336-93de-9d938284eac6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:48.936990 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ce629b-c2d1-4336-93de-9d938284eac6 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:48.968124 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 339/678] 10.222.0.55 () {56 vars in 1148 bytes} [Wed Aug 24 08:34:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:48.970513 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72ac008-16e8-4980-aa5a-30f82f0c305d None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:48.970984 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72ac008-16e8-4980-aa5a-30f82f0c305d None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:48.971445 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72ac008-16e8-4980-aa5a-30f82f0c305d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:49.001916 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-f72ac008-16e8-4980-aa5a-30f82f0c305d None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:49.004190 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-f72ac008-16e8-4980-aa5a-30f82f0c305d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:49.039240 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98db4ad8-98f9-4753-82e3-3a3fc367fca4 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:49.086411 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef976d0-d061-4182-8c6d-ae4462ab57d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:49.087394 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef976d0-d061-4182-8c6d-ae4462ab57d6 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:49.087394 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef976d0-d061-4182-8c6d-ae4462ab57d6 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:49.087639 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef976d0-d061-4182-8c6d-ae4462ab57d6 None admin] PATH_INFO: `/v3/domains` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:49.088085 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef976d0-d061-4182-8c6d-ae4462ab57d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:49.088380 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef976d0-d061-4182-8c6d-ae4462ab57d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:49.089815 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef976d0-d061-4182-8c6d-ae4462ab57d6 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:49.111704 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 340/679] 10.222.0.55 () {56 vars in 1109 bytes} [Wed Aug 24 08:34:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:49.111704 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef976d0-d061-4182-8c6d-ae4462ab57d6 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:49.144892 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 340/680] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:34:49.146597 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72ac008-16e8-4980-aa5a-30f82f0c305d None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:49.220207 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:49.237827 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef976d0-d061-4182-8c6d-ae4462ab57d6 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:49.278064 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f293e42-8733-4d0d-8d2e-f026930c0645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:49.278995 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f293e42-8733-4d0d-8d2e-f026930c0645 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:49.279433 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f293e42-8733-4d0d-8d2e-f026930c0645 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:49.279888 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f293e42-8733-4d0d-8d2e-f026930c0645 None admin] PATH_INFO: `/v3/users/swift` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:49.288898 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f293e42-8733-4d0d-8d2e-f026930c0645 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:49.290663 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f293e42-8733-4d0d-8d2e-f026930c0645 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:49.303829 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-4f293e42-8733-4d0d-8d2e-f026930c0645 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 08:34:49.303829 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:49.303829 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:49.303829 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:49.303829 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:49.303829 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:49.303829 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:49.303829 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:49.303829 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:49.303829 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:49.303829 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:49.303829 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:49.303829 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:34:49.303829 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:34:49.303829 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:34:49.303829 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:34:49.304925 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:49.304925 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:49.304925 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:34:49.304925 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:34:49.304925 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:34:49.304925 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:34:49.304925 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:49.304925 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:49.304925 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:49.304925 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:49.304925 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:49.304925 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:49.304925 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:49.304925 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:49.304925 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:49.304925 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:49.304925 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:49.304925 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:49.305899 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:49.305899 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:49.305899 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:34:49.305899 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:34:49.305899 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:34:49.305899 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:34:49.305899 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:34:49.305899 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:34:49.305899 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 08:34:49.305899 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:34:49.308519 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 341/681] 10.222.0.55 () {56 vars in 1096 bytes} [Wed Aug 24 08:34:49 2022] GET /identity/v3/users/swift => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:49.310287 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f293e42-8733-4d0d-8d2e-f026930c0645 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:49.363823 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:49.371326 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72ac008-16e8-4980-aa5a-30f82f0c305d None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:49.426446 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f293e42-8733-4d0d-8d2e-f026930c0645 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:49.444039 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ea7027-d1f4-4b16-91a3-f8577702eb4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:49.445236 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ea7027-d1f4-4b16-91a3-f8577702eb4a None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:49.445803 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ea7027-d1f4-4b16-91a3-f8577702eb4a None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:49.446346 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ea7027-d1f4-4b16-91a3-f8577702eb4a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:49.448046 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ea7027-d1f4-4b16-91a3-f8577702eb4a None admin] RBAC: Adding query filter params (role.id=bab90e02eebc473e9b911c8660ed9b76, scope.project.id=420b18725f2342159646fb86438318fd, user.id=28b58388d1664510b0b507323bca5ae8) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:49.448676 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ea7027-d1f4-4b16-91a3-f8577702eb4a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:49.451960 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ea7027-d1f4-4b16-91a3-f8577702eb4a None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:49.464506 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 341/682] 10.222.0.55 () {56 vars in 1374 bytes} [Wed Aug 24 08:34:49 2022] GET /identity/v3/role_assignments?user.id=28b58388d1664510b0b507323bca5ae8&scope.project.id=420b18725f2342159646fb86438318fd&role.id=bab90e02eebc473e9b911c8660ed9b76 => generated 608 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:49.467017 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ea7027-d1f4-4b16-91a3-f8577702eb4a None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:49.468658 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48f6b7b-ef56-448d-9a0a-caacb882ff74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:49.469594 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48f6b7b-ef56-448d-9a0a-caacb882ff74 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:49.470060 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48f6b7b-ef56-448d-9a0a-caacb882ff74 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:49.470507 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48f6b7b-ef56-448d-9a0a-caacb882ff74 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:49.479082 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48f6b7b-ef56-448d-9a0a-caacb882ff74 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:49.481586 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48f6b7b-ef56-448d-9a0a-caacb882ff74 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:49.494918 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-c48f6b7b-ef56-448d-9a0a-caacb882ff74 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:34:49.494918 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:49.494918 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:49.494918 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:49.494918 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:49.494918 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:49.494918 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:49.494918 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:49.494918 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:49.494918 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:49.494918 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:49.494918 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:49.494918 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:34:49.494918 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:34:49.494918 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:34:49.495999 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:34:49.495999 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:49.495999 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:49.495999 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:49.495999 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:49.495999 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:49.495999 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:49.495999 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:49.495999 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:49.495999 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:49.495999 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:49.495999 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:49.495999 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:49.495999 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:49.495999 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:49.495999 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:49.495999 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:49.497021 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:34:49.497021 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:34:49.497021 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:34:49.497021 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:34:49.497021 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:34:49.497021 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:34:49.497021 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:34:49.497021 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:34:49.499676 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 342/683] 10.222.0.55 () {56 vars in 1111 bytes} [Wed Aug 24 08:34:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:49.510116 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48f6b7b-ef56-448d-9a0a-caacb882ff74 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:49.523040 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:49.676415 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48f6b7b-ef56-448d-9a0a-caacb882ff74 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:49.695114 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ea7027-d1f4-4b16-91a3-f8577702eb4a None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:49.741220 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5912555-8110-4b78-aa9e-76605d4ad2b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:49.742008 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5912555-8110-4b78-aa9e-76605d4ad2b2 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:49.742374 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5912555-8110-4b78-aa9e-76605d4ad2b2 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:49.742739 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5912555-8110-4b78-aa9e-76605d4ad2b2 None admin] PATH_INFO: `/v3/users` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:49.743579 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5912555-8110-4b78-aa9e-76605d4ad2b2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:49.743975 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5912555-8110-4b78-aa9e-76605d4ad2b2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:49.744142 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48a9309-abca-41e7-adeb-23a9d4c1097c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:49.745233 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48a9309-abca-41e7-adeb-23a9d4c1097c None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:49.747821 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5912555-8110-4b78-aa9e-76605d4ad2b2 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:49.748021 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48a9309-abca-41e7-adeb-23a9d4c1097c None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:49.748021 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48a9309-abca-41e7-adeb-23a9d4c1097c None admin] PATH_INFO: `/v3/projects` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:49.748021 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48a9309-abca-41e7-adeb-23a9d4c1097c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:49.748451 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48a9309-abca-41e7-adeb-23a9d4c1097c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:49.750968 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48a9309-abca-41e7-adeb-23a9d4c1097c None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:49.769473 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 342/684] 10.222.0.55 () {56 vars in 1135 bytes} [Wed Aug 24 08:34:49 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:49.770502 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5912555-8110-4b78-aa9e-76605d4ad2b2 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:49.785643 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 343/685] 10.222.0.55 () {56 vars in 1148 bytes} [Wed Aug 24 08:34:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:49.787535 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48a9309-abca-41e7-adeb-23a9d4c1097c None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:49.796808 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:49.930856 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:49.940931 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5912555-8110-4b78-aa9e-76605d4ad2b2 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:49.999675 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48a9309-abca-41e7-adeb-23a9d4c1097c None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:50.021802 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bcfdf8-2593-46f2-a08e-3a562dd1f2ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:50.023271 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bcfdf8-2593-46f2-a08e-3a562dd1f2ae None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:50.023942 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bcfdf8-2593-46f2-a08e-3a562dd1f2ae None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:50.024676 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bcfdf8-2593-46f2-a08e-3a562dd1f2ae None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:50.046028 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bcfdf8-2593-46f2-a08e-3a562dd1f2ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:34:50.047968 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adcfbdf-7f30-42ad-a754-0ff166e3ef25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:50.048780 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adcfbdf-7f30-42ad-a754-0ff166e3ef25 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:50.049155 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adcfbdf-7f30-42ad-a754-0ff166e3ef25 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:50.049520 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adcfbdf-7f30-42ad-a754-0ff166e3ef25 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:50.057191 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-58bcfdf8-2593-46f2-a08e-3a562dd1f2ae None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:34:50.057191 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:50.057191 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:50.057191 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:50.057191 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:50.057191 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:50.057191 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:50.057191 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:50.057191 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:50.057191 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:50.057191 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:50.057191 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:50.057191 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:34:50.057191 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:34:50.057191 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:34:50.057191 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:34:50.059394 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:34:50.059394 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:34:50.059394 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:34:50.059394 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:34:50.059394 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:50.059394 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:50.059394 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:50.059394 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:50.059394 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:50.059394 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:50.059394 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:50.059394 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:50.059394 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:50.059394 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:50.059394 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:50.059394 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:50.061358 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:50.061358 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:50.061358 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:50.061358 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:50.061358 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:34:50.061358 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:34:50.061358 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:34:50.061358 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:34:50.061358 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:34:50.061358 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:34:50.061358 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:34:50.061358 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:34:50.065768 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 343/686] 10.222.0.55 () {56 vars in 1096 bytes} [Wed Aug 24 08:34:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:50.067732 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bcfdf8-2593-46f2-a08e-3a562dd1f2ae None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:50.154206 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adcfbdf-7f30-42ad-a754-0ff166e3ef25 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:50.158170 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adcfbdf-7f30-42ad-a754-0ff166e3ef25 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:50.227554 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:50.258136 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:50.271567 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bcfdf8-2593-46f2-a08e-3a562dd1f2ae None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:50.306355 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 344/687] 10.222.0.55 () {58 vars in 1282 bytes} [Wed Aug 24 08:34:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 520 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:34:50.308743 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adcfbdf-7f30-42ad-a754-0ff166e3ef25 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:50.339729 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943e57d5-63b0-4345-94dd-96f480c82629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:50.341048 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943e57d5-63b0-4345-94dd-96f480c82629 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:50.341516 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943e57d5-63b0-4345-94dd-96f480c82629 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:50.342016 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943e57d5-63b0-4345-94dd-96f480c82629 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:50.343396 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943e57d5-63b0-4345-94dd-96f480c82629 None admin] RBAC: Adding query filter params (role.id=bab90e02eebc473e9b911c8660ed9b76, scope.project.id=420b18725f2342159646fb86438318fd, user.id=fbbb9ac667f04c35a63b2689df18efed) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:50.343959 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943e57d5-63b0-4345-94dd-96f480c82629 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:50.346854 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943e57d5-63b0-4345-94dd-96f480c82629 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:50.357584 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 344/688] 10.222.0.55 () {56 vars in 1374 bytes} [Wed Aug 24 08:34:50 2022] GET /identity/v3/role_assignments?user.id=fbbb9ac667f04c35a63b2689df18efed&scope.project.id=420b18725f2342159646fb86438318fd&role.id=bab90e02eebc473e9b911c8660ed9b76 => generated 608 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:50.358485 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943e57d5-63b0-4345-94dd-96f480c82629 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:50.586033 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adcfbdf-7f30-42ad-a754-0ff166e3ef25 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:50.653776 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943e57d5-63b0-4345-94dd-96f480c82629 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:50.656667 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0e7162-f542-431a-87ad-308e33efa09a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:50.657997 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0e7162-f542-431a-87ad-308e33efa09a None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:50.658737 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0e7162-f542-431a-87ad-308e33efa09a None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:50.659296 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0e7162-f542-431a-87ad-308e33efa09a None admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:50.660479 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0e7162-f542-431a-87ad-308e33efa09a None admin] RBAC: Adding query filter params (name=admin) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:50.661036 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0e7162-f542-431a-87ad-308e33efa09a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:50.664579 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0e7162-f542-431a-87ad-308e33efa09a None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:50.684748 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 345/689] 10.222.0.55 () {56 vars in 1099 bytes} [Wed Aug 24 08:34:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:50.688665 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0e7162-f542-431a-87ad-308e33efa09a None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:50.730056 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5301cb9-7177-47a6-8649-639f407cec1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:50.731970 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5301cb9-7177-47a6-8649-639f407cec1e None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:50.732730 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5301cb9-7177-47a6-8649-639f407cec1e None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:50.733561 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5301cb9-7177-47a6-8649-639f407cec1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:50.745316 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:50.755619 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5301cb9-7177-47a6-8649-639f407cec1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:50.759323 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5301cb9-7177-47a6-8649-639f407cec1e None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:50.772803 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:50.813807 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-a5301cb9-7177-47a6-8649-639f407cec1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:34:50.813807 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:50.813807 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:50.813807 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:50.813807 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:50.813807 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:50.813807 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:50.813807 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:50.813807 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:50.813807 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:50.813807 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:50.813807 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:50.813807 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:34:50.813807 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:34:50.813807 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:34:50.813807 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:34:50.815659 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:50.815659 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:50.815659 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:50.815659 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:50.815659 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:50.815659 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:50.815659 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:50.815659 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:50.815659 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:50.815659 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:50.815659 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:50.815659 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:50.815659 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:50.815659 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:50.815659 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:50.815659 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:50.815659 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:34:50.817514 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:34:50.817514 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:34:50.817514 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:34:50.823283 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 345/690] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:34:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 460 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:50.829611 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5301cb9-7177-47a6-8649-639f407cec1e None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:50.864680 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0e7162-f542-431a-87ad-308e33efa09a None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:50.909478 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5078164e-3c53-4cc3-adf7-68f798e7d2d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:50.910157 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5078164e-3c53-4cc3-adf7-68f798e7d2d1 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:50.910590 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5078164e-3c53-4cc3-adf7-68f798e7d2d1 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:50.911062 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5078164e-3c53-4cc3-adf7-68f798e7d2d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:50.954302 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5078164e-3c53-4cc3-adf7-68f798e7d2d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:50.957403 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5078164e-3c53-4cc3-adf7-68f798e7d2d1 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:50.986722 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-5078164e-3c53-4cc3-adf7-68f798e7d2d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:34:50.986722 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:50.986722 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:50.986722 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:50.986722 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:50.986722 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:50.986722 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:50.986722 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:50.986722 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:50.986722 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:50.986722 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:50.986722 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:50.986722 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:34:50.986722 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:34:50.986722 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:34:50.986722 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:34:50.987782 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:50.987782 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:50.987782 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:50.987782 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:50.987782 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:50.987782 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:50.987782 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:50.987782 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:50.987782 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:50.987782 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:50.987782 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:50.987782 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:50.987782 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:50.987782 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:50.987782 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:50.987782 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:50.987782 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:34:50.988758 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:34:50.988758 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:34:50.988758 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:34:50.988758 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 346/691] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:34:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:50.997542 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5078164e-3c53-4cc3-adf7-68f798e7d2d1 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:51.080283 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5301cb9-7177-47a6-8649-639f407cec1e None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:51.147104 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:51.157299 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b864e24-0a2e-47c0-8f04-17d78c95eeec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:51.169191 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b864e24-0a2e-47c0-8f04-17d78c95eeec None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:51.169191 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b864e24-0a2e-47c0-8f04-17d78c95eeec None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:51.169191 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b864e24-0a2e-47c0-8f04-17d78c95eeec None admin] PATH_INFO: `/v3/domains` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:51.169191 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b864e24-0a2e-47c0-8f04-17d78c95eeec None admin] RBAC: Adding query filter params (name=Default) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:51.169191 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b864e24-0a2e-47c0-8f04-17d78c95eeec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:51.169191 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b864e24-0a2e-47c0-8f04-17d78c95eeec None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:51.169900 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5078164e-3c53-4cc3-adf7-68f798e7d2d1 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:51.190455 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 346/692] 10.222.0.55 () {56 vars in 1109 bytes} [Wed Aug 24 08:34:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:51.196564 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b864e24-0a2e-47c0-8f04-17d78c95eeec None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:51.218509 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c4b8a6-0a9c-417d-9c06-2e7ab4f5077d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:51.220245 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c4b8a6-0a9c-417d-9c06-2e7ab4f5077d None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:51.221400 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c4b8a6-0a9c-417d-9c06-2e7ab4f5077d None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:51.222465 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c4b8a6-0a9c-417d-9c06-2e7ab4f5077d None admin] PATH_INFO: `/v3/domains` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:51.224129 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c4b8a6-0a9c-417d-9c06-2e7ab4f5077d None admin] RBAC: Adding query filter params (name=Default) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:51.224638 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c4b8a6-0a9c-417d-9c06-2e7ab4f5077d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:51.227884 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c4b8a6-0a9c-417d-9c06-2e7ab4f5077d None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:51.261037 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 347/693] 10.222.0.55 () {56 vars in 1109 bytes} [Wed Aug 24 08:34:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:51.278662 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c4b8a6-0a9c-417d-9c06-2e7ab4f5077d None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:51.320942 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:51.403492 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b864e24-0a2e-47c0-8f04-17d78c95eeec None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:51.461389 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3070d409-ce8b-40e6-be60-997e6c0b277e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:51.462055 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3070d409-ce8b-40e6-be60-997e6c0b277e None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:51.462329 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3070d409-ce8b-40e6-be60-997e6c0b277e None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:51.462599 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3070d409-ce8b-40e6-be60-997e6c0b277e None admin] PATH_INFO: `/v3/projects/service` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:51.483376 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3070d409-ce8b-40e6-be60-997e6c0b277e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:51.485115 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3070d409-ce8b-40e6-be60-997e6c0b277e None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:51.511285 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-3070d409-ce8b-40e6-be60-997e6c0b277e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:34:51.511285 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:51.511285 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:51.511285 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:51.511285 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:51.511285 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:51.511285 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:51.511285 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:51.511285 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:51.511285 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:51.511285 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:51.511285 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:51.511285 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:34:51.511285 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:34:51.511285 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:34:51.512293 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:34:51.512293 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:51.512293 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:51.512293 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:51.512293 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:51.512293 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:51.512293 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:51.512293 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:51.512293 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:51.512293 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:51.512293 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:51.512293 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:51.512293 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:51.512293 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:51.512293 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:51.512293 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:51.512293 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:51.513307 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:34:51.513307 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:34:51.513307 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:34:51.513307 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:34:51.513307 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:34:51.513307 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:34:51.513307 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:34:51.513307 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:34:51.515028 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c4b8a6-0a9c-417d-9c06-2e7ab4f5077d None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:51.524580 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 347/694] 10.222.0.55 () {56 vars in 1111 bytes} [Wed Aug 24 08:34:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:34:51.525571 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3070d409-ce8b-40e6-be60-997e6c0b277e None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:51.575809 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd25755f-39d0-4ec0-900b-7cbe779d695e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:51.576847 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd25755f-39d0-4ec0-900b-7cbe779d695e None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:51.577339 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd25755f-39d0-4ec0-900b-7cbe779d695e None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:51.577802 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd25755f-39d0-4ec0-900b-7cbe779d695e None admin] PATH_INFO: `/v3/users/nova` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:51.586912 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd25755f-39d0-4ec0-900b-7cbe779d695e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:51.588751 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd25755f-39d0-4ec0-900b-7cbe779d695e None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:51.592413 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:51.603671 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-fd25755f-39d0-4ec0-900b-7cbe779d695e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 08:34:51.603671 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:51.603671 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:51.603671 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:51.603671 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:51.603671 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:51.603671 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:51.603671 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:51.603671 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:51.603671 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:51.603671 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:51.603671 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:51.603671 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:34:51.603671 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:34:51.603671 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:34:51.603671 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:34:51.607555 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:51.607555 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:51.607555 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:34:51.607555 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:34:51.607555 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:34:51.607555 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:34:51.607555 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:51.607555 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:51.607555 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:51.607555 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:51.607555 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:51.607555 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:51.607555 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:51.607555 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:51.607555 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:51.607555 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:51.607555 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:51.607555 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:51.608859 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:51.608859 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:51.608859 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:34:51.608859 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:34:51.608859 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:34:51.608859 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:34:51.608859 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:34:51.608859 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:34:51.608859 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 08:34:51.608859 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:34:51.608859 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 348/695] 10.222.0.55 () {56 vars in 1093 bytes} [Wed Aug 24 08:34:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:51.611008 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1867055f-39ec-44b5-8191-1440b2fde9df None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:51.611443 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1867055f-39ec-44b5-8191-1440b2fde9df None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:51.611856 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1867055f-39ec-44b5-8191-1440b2fde9df None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:51.617975 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 349/696] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:34:51 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:34:51.619259 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1867055f-39ec-44b5-8191-1440b2fde9df None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:51.707739 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3070d409-ce8b-40e6-be60-997e6c0b277e None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:51.742657 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:51.745959 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1867055f-39ec-44b5-8191-1440b2fde9df None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:51.746564 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0471f616-b231-414a-a198-ff1a93044a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:51.747270 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0471f616-b231-414a-a198-ff1a93044a9e None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:51.757752 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0471f616-b231-414a-a198-ff1a93044a9e None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:51.757752 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0471f616-b231-414a-a198-ff1a93044a9e None admin] PATH_INFO: `/v3/projects` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:51.757752 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0471f616-b231-414a-a198-ff1a93044a9e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:51.757752 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0471f616-b231-414a-a198-ff1a93044a9e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:51.757752 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0471f616-b231-414a-a198-ff1a93044a9e None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:51.770127 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 348/697] 10.222.0.55 () {56 vars in 1148 bytes} [Wed Aug 24 08:34:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:51.770127 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d767c4-2ead-4afa-87dc-3851e9d3d719 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:51.770127 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d767c4-2ead-4afa-87dc-3851e9d3d719 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:51.770127 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d767c4-2ead-4afa-87dc-3851e9d3d719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:51.788368 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-83d767c4-2ead-4afa-87dc-3851e9d3d719 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:51.789632 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-83d767c4-2ead-4afa-87dc-3851e9d3d719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:51.795930 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f322a6e-a99d-4723-b0cb-2aa668245ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:51.796847 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f322a6e-a99d-4723-b0cb-2aa668245ed8 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:51.797295 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f322a6e-a99d-4723-b0cb-2aa668245ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:51.797716 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f322a6e-a99d-4723-b0cb-2aa668245ed8 None admin] PATH_INFO: `/v3/users` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:51.798627 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f322a6e-a99d-4723-b0cb-2aa668245ed8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:51.799087 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f322a6e-a99d-4723-b0cb-2aa668245ed8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:51.800731 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f322a6e-a99d-4723-b0cb-2aa668245ed8 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:51.821266 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 350/698] 10.222.0.55 () {56 vars in 1133 bytes} [Wed Aug 24 08:34:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:51.822593 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f322a6e-a99d-4723-b0cb-2aa668245ed8 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:51.881275 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 349/699] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:34:51.881876 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d767c4-2ead-4afa-87dc-3851e9d3d719 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:51.923509 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:51.987187 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f322a6e-a99d-4723-b0cb-2aa668245ed8 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:52.029273 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be00073-04c9-49b4-826f-4a2213ada774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:52.030002 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be00073-04c9-49b4-826f-4a2213ada774 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:52.030281 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be00073-04c9-49b4-826f-4a2213ada774 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:52.030550 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be00073-04c9-49b4-826f-4a2213ada774 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:52.031288 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be00073-04c9-49b4-826f-4a2213ada774 None admin] RBAC: Adding query filter params (role.id=bab90e02eebc473e9b911c8660ed9b76, scope.project.id=420b18725f2342159646fb86438318fd, user.id=a0ebb5ccabef45378f0c1da69721e051) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:52.031590 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be00073-04c9-49b4-826f-4a2213ada774 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:52.033054 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be00073-04c9-49b4-826f-4a2213ada774 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:52.040292 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 351/700] 10.222.0.55 () {56 vars in 1374 bytes} [Wed Aug 24 08:34:51 2022] GET /identity/v3/role_assignments?user.id=a0ebb5ccabef45378f0c1da69721e051&scope.project.id=420b18725f2342159646fb86438318fd&role.id=bab90e02eebc473e9b911c8660ed9b76 => generated 608 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:52.041652 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ea05eb-1b9f-46c0-9bb6-4cca331b0749 None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:52.041940 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ea05eb-1b9f-46c0-9bb6-4cca331b0749 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:52.042212 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ea05eb-1b9f-46c0-9bb6-4cca331b0749 None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:52.044393 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 352/701] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:34:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:34:52.054131 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e225ad6-0ce0-496f-a795-36a683649677 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:52.054468 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e225ad6-0ce0-496f-a795-36a683649677 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:52.054690 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e225ad6-0ce0-496f-a795-36a683649677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:52.060666 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d767c4-2ead-4afa-87dc-3851e9d3d719 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:52.073018 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:52.106240 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-8e225ad6-0ce0-496f-a795-36a683649677 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:52.107732 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-8e225ad6-0ce0-496f-a795-36a683649677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:52.134739 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d14241-51c4-4e36-a1b4-f0d6dd764761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:52.136387 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d14241-51c4-4e36-a1b4-f0d6dd764761 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:52.137875 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d14241-51c4-4e36-a1b4-f0d6dd764761 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:52.139520 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d14241-51c4-4e36-a1b4-f0d6dd764761 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:52.269197 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d14241-51c4-4e36-a1b4-f0d6dd764761 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:52.270604 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d14241-51c4-4e36-a1b4-f0d6dd764761 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:52.280120 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 353/702] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:34:52.281784 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e225ad6-0ce0-496f-a795-36a683649677 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:52.367459 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:52.403791 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 350/703] 10.222.0.55 () {58 vars in 1282 bytes} [Wed Aug 24 08:34:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 522 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:34:52.409968 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d14241-51c4-4e36-a1b4-f0d6dd764761 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:52.420517 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:52.429735 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e225ad6-0ce0-496f-a795-36a683649677 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:52.471703 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73886971-0838-4439-9539-18105622d07e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:52.472347 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73886971-0838-4439-9539-18105622d07e None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:52.472653 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73886971-0838-4439-9539-18105622d07e None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:52.472923 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73886971-0838-4439-9539-18105622d07e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:52.486267 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73886971-0838-4439-9539-18105622d07e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:52.487794 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73886971-0838-4439-9539-18105622d07e None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:52.506015 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-73886971-0838-4439-9539-18105622d07e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:34:52.506015 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:52.506015 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:52.506015 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:52.506015 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:52.506015 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:52.506015 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:52.506015 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:52.506015 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:52.506015 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:52.506015 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:52.506015 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:52.506015 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:34:52.506015 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:34:52.506015 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:34:52.506015 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:34:52.507066 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:52.507066 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:52.507066 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:52.507066 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:52.507066 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:52.507066 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:52.507066 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:52.507066 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:52.507066 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:52.507066 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:52.507066 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:52.507066 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:52.507066 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:52.507066 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:52.507066 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:52.507066 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:52.507066 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:34:52.508066 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:34:52.508066 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:34:52.508066 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:34:52.508066 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 354/704] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:34:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:52.523787 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73886971-0838-4439-9539-18105622d07e None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:52.611658 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d14241-51c4-4e36-a1b4-f0d6dd764761 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:52.650973 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73886971-0838-4439-9539-18105622d07e None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:52.667499 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:52.695352 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9851cc4-7232-4855-807d-14e470e50a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:52.696681 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9851cc4-7232-4855-807d-14e470e50a8c None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:52.697244 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9851cc4-7232-4855-807d-14e470e50a8c None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:52.697717 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9851cc4-7232-4855-807d-14e470e50a8c None admin] PATH_INFO: `/v3/domains` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:52.698582 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9851cc4-7232-4855-807d-14e470e50a8c None admin] RBAC: Adding query filter params (name=Default) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:52.699091 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9851cc4-7232-4855-807d-14e470e50a8c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:52.700882 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9851cc4-7232-4855-807d-14e470e50a8c None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:52.714653 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 355/705] 10.222.0.55 () {56 vars in 1109 bytes} [Wed Aug 24 08:34:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:34:52.720071 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54401036-bf37-43f3-bf9e-300ad07559dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:52.721397 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54401036-bf37-43f3-bf9e-300ad07559dd None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:52.721905 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54401036-bf37-43f3-bf9e-300ad07559dd None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:52.723225 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9851cc4-7232-4855-807d-14e470e50a8c None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:52.724144 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54401036-bf37-43f3-bf9e-300ad07559dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:52.754038 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54401036-bf37-43f3-bf9e-300ad07559dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:52.756187 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54401036-bf37-43f3-bf9e-300ad07559dd None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:52.786944 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-54401036-bf37-43f3-bf9e-300ad07559dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:34:52.786944 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:52.786944 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:52.786944 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:52.786944 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:52.786944 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:52.786944 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:52.786944 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:52.786944 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:52.786944 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:52.786944 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:52.786944 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:52.786944 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:34:52.786944 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:34:52.786944 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:34:52.786944 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:34:52.787982 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:52.787982 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:52.787982 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:52.787982 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:52.787982 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:52.787982 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:52.787982 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:52.787982 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:52.787982 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:52.787982 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:52.787982 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:52.787982 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:52.787982 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:52.787982 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:52.787982 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:52.787982 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:52.787982 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:34:52.788970 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:34:52.788970 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:34:52.788970 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:34:52.796623 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 351/706] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:34:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 387 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:52.802839 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54401036-bf37-43f3-bf9e-300ad07559dd None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:52.919733 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:52.932851 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9851cc4-7232-4855-807d-14e470e50a8c None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:52.981912 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54401036-bf37-43f3-bf9e-300ad07559dd None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:52.987174 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38774d53-752e-419d-aab9-ae4cf804255d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:52.987802 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38774d53-752e-419d-aab9-ae4cf804255d None admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:52.988083 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38774d53-752e-419d-aab9-ae4cf804255d None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:52.988354 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38774d53-752e-419d-aab9-ae4cf804255d None admin] PATH_INFO: `/v3/users` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:52.989080 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38774d53-752e-419d-aab9-ae4cf804255d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:52.991202 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38774d53-752e-419d-aab9-ae4cf804255d None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:53.029012 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc18b40-127e-4e56-96b9-22eb702ce613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:53.029627 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc18b40-127e-4e56-96b9-22eb702ce613 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:53.029907 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc18b40-127e-4e56-96b9-22eb702ce613 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:53.030182 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc18b40-127e-4e56-96b9-22eb702ce613 None admin] PATH_INFO: `/v3/domains` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:53.030838 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc18b40-127e-4e56-96b9-22eb702ce613 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:53.031159 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc18b40-127e-4e56-96b9-22eb702ce613 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:53.032530 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc18b40-127e-4e56-96b9-22eb702ce613 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:53.050235 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 352/707] 10.222.0.55 () {56 vars in 1109 bytes} [Wed Aug 24 08:34:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:53.050235 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc18b40-127e-4e56-96b9-22eb702ce613 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:53.102939 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 356/708] 10.222.0.55 () {60 vars in 1132 bytes} [Wed Aug 24 08:34:52 2022] POST /identity/v3/users => generated 254 bytes in 380 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:34:53.103806 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38774d53-752e-419d-aab9-ae4cf804255d None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:53.138064 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:53.234668 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc18b40-127e-4e56-96b9-22eb702ce613 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:53.275305 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:53.305591 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38774d53-752e-419d-aab9-ae4cf804255d None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:53.310772 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad476a8-4adb-4225-a35c-5d4135fd06d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:53.311679 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad476a8-4adb-4225-a35c-5d4135fd06d5 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:53.312111 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad476a8-4adb-4225-a35c-5d4135fd06d5 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:53.312589 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad476a8-4adb-4225-a35c-5d4135fd06d5 None admin] PATH_INFO: `/v3/services/network` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:53.313346 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad476a8-4adb-4225-a35c-5d4135fd06d5 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:53.314918 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad476a8-4adb-4225-a35c-5d4135fd06d5 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:53.327826 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-cad476a8-4adb-4225-a35c-5d4135fd06d5 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 08:34:53.327826 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:53.327826 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 08:34:53.327826 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 08:34:53.327826 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 08:34:53.327826 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 08:34:53.327826 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 08:34:53.327826 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:34:53.327826 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 08:34:53.327826 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:34:53.327826 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:34:53.327826 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:34:53.327826 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:53.327826 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:53.327826 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:53.327826 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:53.327826 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:53.330171 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:53.330171 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:53.330171 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:53.330171 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:53.330171 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:53.330171 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:53.330171 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 08:34:53.330171 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 08:34:53.330171 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 08:34:53.330171 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 08:34:53.330171 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:53.330171 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:53.330171 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:53.330171 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:53.330171 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:53.330171 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:53.331851 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:53.331851 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:53.331851 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:53.331851 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:53.331851 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:53.331851 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:53.331851 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:53.331851 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:53.331851 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:53.331851 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:53.331851 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 08:34:53.331851 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:34:53.331851 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 08:34:53.331851 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:34:53.331851 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 353/709] 10.222.0.55 () {56 vars in 1111 bytes} [Wed Aug 24 08:34:53 2022] GET /identity/v3/services/network => generated 88 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:53.334507 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df1fede-8171-4326-94f3-900761838af1 None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:53.334507 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df1fede-8171-4326-94f3-900761838af1 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:53.334507 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df1fede-8171-4326-94f3-900761838af1 None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:53.334507 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 354/710] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:34:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:34:53.338490 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8c9e09-601d-41f1-893b-7618b242f955 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:53.339957 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8c9e09-601d-41f1-893b-7618b242f955 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:53.339957 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8c9e09-601d-41f1-893b-7618b242f955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:53.370415 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-cb8c9e09-601d-41f1-893b-7618b242f955 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:53.373393 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-cb8c9e09-601d-41f1-893b-7618b242f955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:53.397388 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5cd69c-118a-4d7d-aa22-010bb96f210c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:53.398565 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5cd69c-118a-4d7d-aa22-010bb96f210c None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:53.399066 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5cd69c-118a-4d7d-aa22-010bb96f210c None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:53.399557 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5cd69c-118a-4d7d-aa22-010bb96f210c None admin] PATH_INFO: `/v3/projects/service` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:53.413027 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5cd69c-118a-4d7d-aa22-010bb96f210c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:53.415792 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5cd69c-118a-4d7d-aa22-010bb96f210c None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:53.441251 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-3e5cd69c-118a-4d7d-aa22-010bb96f210c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:34:53.441251 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:53.441251 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:53.441251 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:53.441251 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:53.441251 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:53.441251 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:53.441251 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:53.441251 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:53.441251 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:53.441251 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:53.441251 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:53.441251 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:34:53.441251 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:34:53.441251 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:34:53.442250 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:34:53.442250 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:53.442250 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:53.442250 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:53.442250 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:53.442250 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:53.442250 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:53.442250 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:53.442250 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:53.442250 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:53.442250 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:53.442250 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:53.442250 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:53.442250 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:53.442250 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:53.442250 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:53.442250 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:53.443169 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:34:53.443169 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:34:53.443169 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:34:53.443169 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:34:53.443169 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:34:53.443169 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:34:53.443169 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:34:53.443169 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:34:53.445622 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 357/711] 10.222.0.55 () {56 vars in 1111 bytes} [Wed Aug 24 08:34:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:53.447299 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5cd69c-118a-4d7d-aa22-010bb96f210c None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:53.523512 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 355/712] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 08:34:53.525024 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8c9e09-601d-41f1-893b-7618b242f955 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:53.642237 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:53.679514 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5cd69c-118a-4d7d-aa22-010bb96f210c None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:53.739396 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8c9e09-601d-41f1-893b-7618b242f955 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:53.753344 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e291af6-7616-4be6-97b1-73c103ffeae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:53.755100 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e291af6-7616-4be6-97b1-73c103ffeae7 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:53.755880 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e291af6-7616-4be6-97b1-73c103ffeae7 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:53.756741 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e291af6-7616-4be6-97b1-73c103ffeae7 None admin] PATH_INFO: `/v3/services` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:53.758209 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e291af6-7616-4be6-97b1-73c103ffeae7 None admin] RBAC: Adding query filter params (name=network) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:53.759255 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e291af6-7616-4be6-97b1-73c103ffeae7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:53.761994 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e291af6-7616-4be6-97b1-73c103ffeae7 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:53.768081 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:53.771328 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 358/713] 10.222.0.55 () {56 vars in 1112 bytes} [Wed Aug 24 08:34:53 2022] GET /identity/v3/services?name=network => generated 124 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:53.789469 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e291af6-7616-4be6-97b1-73c103ffeae7 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:53.827140 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ee3daf-664c-41e1-a6a4-a770f175b403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:53.828219 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ee3daf-664c-41e1-a6a4-a770f175b403 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:53.828707 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ee3daf-664c-41e1-a6a4-a770f175b403 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:53.829166 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ee3daf-664c-41e1-a6a4-a770f175b403 None admin] PATH_INFO: `/v3/projects` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:53.832823 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ee3daf-664c-41e1-a6a4-a770f175b403 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:53.833399 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ee3daf-664c-41e1-a6a4-a770f175b403 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:53.836576 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ee3daf-664c-41e1-a6a4-a770f175b403 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:53.867199 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 356/714] 10.222.0.55 () {56 vars in 1148 bytes} [Wed Aug 24 08:34:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:53.877686 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ee3daf-664c-41e1-a6a4-a770f175b403 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:53.974100 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e291af6-7616-4be6-97b1-73c103ffeae7 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:54.012780 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c0db43-1d53-4fcc-9e43-b88a7e72f232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:54.013424 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c0db43-1d53-4fcc-9e43-b88a7e72f232 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:54.013708 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c0db43-1d53-4fcc-9e43-b88a7e72f232 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:54.013978 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c0db43-1d53-4fcc-9e43-b88a7e72f232 None admin] PATH_INFO: `/v3/services` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:54.014651 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c0db43-1d53-4fcc-9e43-b88a7e72f232 None admin] RBAC: Adding query filter params (type=network) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:54.015010 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c0db43-1d53-4fcc-9e43-b88a7e72f232 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:54.016449 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c0db43-1d53-4fcc-9e43-b88a7e72f232 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:54.023576 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 359/715] 10.222.0.55 () {56 vars in 1112 bytes} [Wed Aug 24 08:34:53 2022] GET /identity/v3/services?type=network => generated 124 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:54.024496 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c0db43-1d53-4fcc-9e43-b88a7e72f232 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:54.040982 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:54.079107 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ee3daf-664c-41e1-a6a4-a770f175b403 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:54.146076 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705a86fc-a861-4659-9a41-8e1b6b5dc354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:54.147084 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705a86fc-a861-4659-9a41-8e1b6b5dc354 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:54.147526 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705a86fc-a861-4659-9a41-8e1b6b5dc354 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:54.147993 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705a86fc-a861-4659-9a41-8e1b6b5dc354 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:54.149385 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705a86fc-a861-4659-9a41-8e1b6b5dc354 None admin] RBAC: Adding query filter params (role.id=b870cd1acaa744e1ae1d58b9ac283821, scope.project.id=420b18725f2342159646fb86438318fd, user.id=f17da357e555420799ee681bffbcc268) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:54.149930 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705a86fc-a861-4659-9a41-8e1b6b5dc354 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:54.152533 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705a86fc-a861-4659-9a41-8e1b6b5dc354 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:54.178557 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 357/716] 10.222.0.55 () {56 vars in 1374 bytes} [Wed Aug 24 08:34:53 2022] GET /identity/v3/role_assignments?user.id=f17da357e555420799ee681bffbcc268&scope.project.id=420b18725f2342159646fb86438318fd&role.id=b870cd1acaa744e1ae1d58b9ac283821 => generated 259 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:34:54.203576 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c0db43-1d53-4fcc-9e43-b88a7e72f232 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:54.282650 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee498b4-cda7-4b63-8f1e-575f579d31f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:54.283233 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee498b4-cda7-4b63-8f1e-575f579d31f1 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:54.283517 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee498b4-cda7-4b63-8f1e-575f579d31f1 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:54.283791 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee498b4-cda7-4b63-8f1e-575f579d31f1 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:54.284485 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee498b4-cda7-4b63-8f1e-575f579d31f1 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:54.285967 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee498b4-cda7-4b63-8f1e-575f579d31f1 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:54.305410 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-0ee498b4-cda7-4b63-8f1e-575f579d31f1 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 08:34:54.305410 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:54.305410 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 08:34:54.305410 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 08:34:54.305410 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 08:34:54.305410 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 08:34:54.305410 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 08:34:54.305410 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:34:54.305410 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 08:34:54.305410 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:34:54.305410 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:34:54.305410 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:34:54.305410 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:54.305410 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:54.305410 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:54.305410 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:54.305410 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:54.306916 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:54.306916 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:54.306916 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:54.306916 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:54.306916 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:54.306916 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:54.306916 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 08:34:54.306916 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 08:34:54.306916 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 08:34:54.306916 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 08:34:54.306916 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:54.306916 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:54.306916 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:54.306916 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:54.306916 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:54.306916 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:54.307775 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:54.307775 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:54.307775 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:54.307775 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:54.307775 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:54.307775 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:54.307775 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:54.307775 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:54.307775 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:54.307775 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:54.307775 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 08:34:54.307775 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:34:54.307775 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 08:34:54.307775 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:34:54.307775 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 360/717] 10.222.0.55 () {56 vars in 1129 bytes} [Wed Aug 24 08:34:54 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:54.315467 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705a86fc-a861-4659-9a41-8e1b6b5dc354 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:54.395782 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:54.483532 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705a86fc-a861-4659-9a41-8e1b6b5dc354 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:54.541837 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4daab3-92c4-418c-bfc4-ac8d704431a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:54.542376 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4daab3-92c4-418c-bfc4-ac8d704431a0 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:54.542659 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4daab3-92c4-418c-bfc4-ac8d704431a0 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:54.542932 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4daab3-92c4-418c-bfc4-ac8d704431a0 None admin] PATH_INFO: `/v3/services` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:54.543616 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4daab3-92c4-418c-bfc4-ac8d704431a0 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:54.543922 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4daab3-92c4-418c-bfc4-ac8d704431a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:54.545347 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4daab3-92c4-418c-bfc4-ac8d704431a0 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:54.554995 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 358/718] 10.222.0.55 () {56 vars in 1124 bytes} [Wed Aug 24 08:34:54 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:54.563726 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee498b4-cda7-4b63-8f1e-575f579d31f1 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:54.670866 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:54.711022 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee498b4-cda7-4b63-8f1e-575f579d31f1 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:54.761078 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d462fc0e-9fb9-431c-8f69-32df2886df01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:54.761708 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d462fc0e-9fb9-431c-8f69-32df2886df01 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:54.761991 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d462fc0e-9fb9-431c-8f69-32df2886df01 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:54.762264 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d462fc0e-9fb9-431c-8f69-32df2886df01 None admin] PATH_INFO: `/v3/services` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:54.763467 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d462fc0e-9fb9-431c-8f69-32df2886df01 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:54.763772 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d462fc0e-9fb9-431c-8f69-32df2886df01 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:54.765225 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d462fc0e-9fb9-431c-8f69-32df2886df01 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:54.771011 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 361/719] 10.222.0.55 () {56 vars in 1124 bytes} [Wed Aug 24 08:34:54 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:54.833249 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8a90ce-f3d3-472f-b465-ee44cd5a28f7 None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:54.833826 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8a90ce-f3d3-472f-b465-ee44cd5a28f7 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:54.834082 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8a90ce-f3d3-472f-b465-ee44cd5a28f7 None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:54.835990 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 359/720] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:34:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:34:54.842707 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c94bca4-5368-4e75-abd2-2c77e77baf07 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:54.843056 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c94bca4-5368-4e75-abd2-2c77e77baf07 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:54.843262 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c94bca4-5368-4e75-abd2-2c77e77baf07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:54.868915 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-8c94bca4-5368-4e75-abd2-2c77e77baf07 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:54.870228 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-8c94bca4-5368-4e75-abd2-2c77e77baf07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:54.966889 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 362/721] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:34:55.247892 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8a90ce-f3d3-472f-b465-ee44cd5a28f7 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:55.443476 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:55.458087 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8a90ce-f3d3-472f-b465-ee44cd5a28f7 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:55.537458 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c805492-2aac-499b-b6d3-d0744c8b4b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:55.538554 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c805492-2aac-499b-b6d3-d0744c8b4b8e None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:55.539048 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c805492-2aac-499b-b6d3-d0744c8b4b8e None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:55.539541 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c805492-2aac-499b-b6d3-d0744c8b4b8e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:55.555189 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c805492-2aac-499b-b6d3-d0744c8b4b8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:34:55.565473 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-9c805492-2aac-499b-b6d3-d0744c8b4b8e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:34:55.565473 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:55.565473 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:55.565473 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:55.565473 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:55.565473 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:55.565473 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:55.565473 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:55.565473 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:55.565473 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:55.565473 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:55.565473 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:55.565473 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:34:55.565473 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:34:55.565473 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:34:55.565473 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:34:55.566486 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:34:55.566486 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:34:55.566486 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:34:55.566486 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:34:55.566486 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:55.566486 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:55.566486 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:55.566486 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:55.566486 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:55.566486 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:55.566486 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:55.566486 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:55.566486 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:55.566486 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:55.566486 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:55.566486 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:55.567761 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:55.567761 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:55.567761 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:55.567761 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:55.567761 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:34:55.567761 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:34:55.567761 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:34:55.567761 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:34:55.567761 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:34:55.567761 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:34:55.567761 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:34:55.567761 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:34:55.567761 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 360/722] 10.222.0.55 () {56 vars in 1096 bytes} [Wed Aug 24 08:34:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:55.577015 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c94bca4-5368-4e75-abd2-2c77e77baf07 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:55.672160 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:55.713565 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c94bca4-5368-4e75-abd2-2c77e77baf07 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:55.765417 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0dd580-f92a-4878-8a45-f38db1ee8a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:55.766111 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0dd580-f92a-4878-8a45-f38db1ee8a07 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:55.766399 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0dd580-f92a-4878-8a45-f38db1ee8a07 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:55.766679 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0dd580-f92a-4878-8a45-f38db1ee8a07 None admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:55.767376 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0dd580-f92a-4878-8a45-f38db1ee8a07 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:55.767682 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0dd580-f92a-4878-8a45-f38db1ee8a07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:55.769132 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0dd580-f92a-4878-8a45-f38db1ee8a07 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:55.779062 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 363/723] 10.222.0.55 () {56 vars in 1099 bytes} [Wed Aug 24 08:34:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:55.784572 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c805492-2aac-499b-b6d3-d0744c8b4b8e None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:55.903744 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2d2b7b-7ec7-4eab-9740-07e42f49c80d None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:55.904373 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2d2b7b-7ec7-4eab-9740-07e42f49c80d None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:55.904893 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2d2b7b-7ec7-4eab-9740-07e42f49c80d None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:55.906712 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 364/724] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:34:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:34:55.912566 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973c0c27-73af-4279-ac20-f8b25374b960 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:55.912889 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973c0c27-73af-4279-ac20-f8b25374b960 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:55.913146 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973c0c27-73af-4279-ac20-f8b25374b960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:55.961977 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-973c0c27-73af-4279-ac20-f8b25374b960 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:55.964527 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-973c0c27-73af-4279-ac20-f8b25374b960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:56.027574 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c805492-2aac-499b-b6d3-d0744c8b4b8e None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:56.102462 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35cda57-da3f-42e2-bd2d-f85ce8a1ebb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:56.103104 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35cda57-da3f-42e2-bd2d-f85ce8a1ebb4 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:56.103380 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35cda57-da3f-42e2-bd2d-f85ce8a1ebb4 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:56.103647 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35cda57-da3f-42e2-bd2d-f85ce8a1ebb4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:56.112300 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 365/725] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 08:34:56.131119 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35cda57-da3f-42e2-bd2d-f85ce8a1ebb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:56.132893 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35cda57-da3f-42e2-bd2d-f85ce8a1ebb4 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:56.157603 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-e35cda57-da3f-42e2-bd2d-f85ce8a1ebb4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:34:56.157603 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:56.157603 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:56.157603 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:56.157603 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:56.157603 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:56.157603 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:56.157603 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:56.157603 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:56.157603 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:56.157603 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:56.157603 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:56.157603 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:34:56.157603 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:34:56.157603 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:34:56.157603 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:34:56.159918 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:56.159918 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:56.159918 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:56.159918 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:56.159918 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:56.159918 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:56.159918 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:56.159918 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:56.159918 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:56.159918 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:56.159918 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:56.159918 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:56.159918 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:56.159918 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:56.159918 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:56.159918 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:56.159918 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:34:56.161948 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:34:56.161948 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:34:56.161948 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:34:56.161948 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 361/726] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:34:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 375 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:56.165694 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973c0c27-73af-4279-ac20-f8b25374b960 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:56.321025 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:56.393388 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973c0c27-73af-4279-ac20-f8b25374b960 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:56.452233 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35cda57-da3f-42e2-bd2d-f85ce8a1ebb4 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:56.463101 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca0976d-1233-49d9-bdbb-8c222fea49eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:56.463718 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca0976d-1233-49d9-bdbb-8c222fea49eb None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:56.464000 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca0976d-1233-49d9-bdbb-8c222fea49eb None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:56.464275 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca0976d-1233-49d9-bdbb-8c222fea49eb None admin] PATH_INFO: `/v3/domains` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:56.464963 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca0976d-1233-49d9-bdbb-8c222fea49eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:56.465330 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca0976d-1233-49d9-bdbb-8c222fea49eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:56.467265 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca0976d-1233-49d9-bdbb-8c222fea49eb None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:56.480886 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 366/727] 10.222.0.55 () {56 vars in 1109 bytes} [Wed Aug 24 08:34:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:56.493629 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca0976d-1233-49d9-bdbb-8c222fea49eb None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:56.494254 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:56.661700 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35cda57-da3f-42e2-bd2d-f85ce8a1ebb4 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:56.706351 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca0976d-1233-49d9-bdbb-8c222fea49eb None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:56.723473 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a819322-78c5-4ee5-91e3-870413f3e0a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:56.724511 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a819322-78c5-4ee5-91e3-870413f3e0a4 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:56.725004 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a819322-78c5-4ee5-91e3-870413f3e0a4 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:56.725457 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a819322-78c5-4ee5-91e3-870413f3e0a4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:56.760200 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a819322-78c5-4ee5-91e3-870413f3e0a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:34:56.771440 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:56.780719 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-8a819322-78c5-4ee5-91e3-870413f3e0a4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:34:56.780719 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:56.780719 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:56.780719 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:56.780719 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:56.780719 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:56.780719 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:56.780719 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:56.780719 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:56.780719 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:56.780719 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:56.780719 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:56.780719 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:34:56.780719 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:34:56.780719 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:34:56.780719 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:34:56.782723 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:34:56.782723 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:34:56.782723 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:34:56.782723 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:34:56.782723 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:56.782723 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:56.782723 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:56.782723 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:56.782723 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:56.782723 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:56.782723 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:56.782723 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:56.782723 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:56.782723 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:56.782723 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:56.782723 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:56.783655 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:56.783655 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:56.783655 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:56.783655 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:56.783655 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:34:56.783655 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:34:56.783655 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:34:56.783655 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:34:56.783655 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:34:56.783655 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:34:56.783655 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:34:56.783655 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:34:56.785415 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 362/728] 10.222.0.55 () {56 vars in 1102 bytes} [Wed Aug 24 08:34:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:56.792206 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a819322-78c5-4ee5-91e3-870413f3e0a4 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:56.821880 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610826d6-62eb-4c16-8d15-a558d26fc496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:56.822899 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610826d6-62eb-4c16-8d15-a558d26fc496 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:56.823385 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610826d6-62eb-4c16-8d15-a558d26fc496 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:56.823876 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610826d6-62eb-4c16-8d15-a558d26fc496 None admin] PATH_INFO: `/v3/users/swift` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:56.845889 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610826d6-62eb-4c16-8d15-a558d26fc496 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:56.849557 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610826d6-62eb-4c16-8d15-a558d26fc496 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:56.877468 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-610826d6-62eb-4c16-8d15-a558d26fc496 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 08:34:56.877468 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:56.877468 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:56.877468 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:56.877468 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:56.877468 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:56.877468 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:56.877468 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:56.877468 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:56.877468 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:56.877468 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:56.877468 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:56.877468 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:34:56.877468 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:34:56.877468 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:34:56.877468 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:34:56.878557 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:56.878557 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:56.878557 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:34:56.878557 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:34:56.878557 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:34:56.878557 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:34:56.878557 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:56.878557 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:56.878557 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:56.878557 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:56.878557 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:56.878557 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:56.878557 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:56.878557 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:56.878557 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:56.878557 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:56.878557 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:56.878557 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:56.879837 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:56.879837 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:56.879837 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:34:56.879837 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:34:56.879837 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:34:56.879837 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:34:56.879837 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:34:56.879837 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:34:56.879837 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 08:34:56.879837 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:34:56.880401 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 367/729] 10.222.0.55 () {56 vars in 1096 bytes} [Wed Aug 24 08:34:56 2022] GET /identity/v3/users/swift => generated 83 bytes in 387 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:56.882413 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31763ce0-62ce-4282-86fe-d04030bf289f None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:56.882908 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31763ce0-62ce-4282-86fe-d04030bf289f None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:56.883395 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31763ce0-62ce-4282-86fe-d04030bf289f None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:56.886066 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 368/730] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:34:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:34:56.888956 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d29d2ec-c9e7-44ab-ae57-0a4c127cfdb1 None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:56.889427 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d29d2ec-c9e7-44ab-ae57-0a4c127cfdb1 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:56.889873 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d29d2ec-c9e7-44ab-ae57-0a4c127cfdb1 None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:56.892812 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 369/731] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:34:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:34:56.894317 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d29d2ec-c9e7-44ab-ae57-0a4c127cfdb1 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:56.914296 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:56.953619 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a819322-78c5-4ee5-91e3-870413f3e0a4 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:56.993649 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67db1bcc-21ad-4774-8ec4-74fb44e21b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:56.994274 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67db1bcc-21ad-4774-8ec4-74fb44e21b56 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:56.994553 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67db1bcc-21ad-4774-8ec4-74fb44e21b56 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:56.994823 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67db1bcc-21ad-4774-8ec4-74fb44e21b56 None admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:56.995490 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67db1bcc-21ad-4774-8ec4-74fb44e21b56 None admin] RBAC: Adding query filter params (name=service) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:56.995806 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67db1bcc-21ad-4774-8ec4-74fb44e21b56 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:56.997797 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67db1bcc-21ad-4774-8ec4-74fb44e21b56 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:57.007279 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 363/732] 10.222.0.55 () {56 vars in 1103 bytes} [Wed Aug 24 08:34:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:57.008605 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25f8b61-e2f9-4062-8157-a28030933cfb None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:57.008896 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25f8b61-e2f9-4062-8157-a28030933cfb None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:57.009174 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25f8b61-e2f9-4062-8157-a28030933cfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:57.029418 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-b25f8b61-e2f9-4062-8157-a28030933cfb None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:57.030668 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-b25f8b61-e2f9-4062-8157-a28030933cfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:57.096022 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d29d2ec-c9e7-44ab-ae57-0a4c127cfdb1 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:57.138502 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 364/733] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:34:57.139924 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac89e828-3099-48c7-a2b7-e7103084765d None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:57.140312 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac89e828-3099-48c7-a2b7-e7103084765d None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:57.140734 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac89e828-3099-48c7-a2b7-e7103084765d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:57.163353 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c64207-1b4d-4923-951b-161a018e6738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:57.164663 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c64207-1b4d-4923-951b-161a018e6738 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:57.165356 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c64207-1b4d-4923-951b-161a018e6738 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:57.166047 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c64207-1b4d-4923-951b-161a018e6738 None admin] PATH_INFO: `/v3/users` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:57.167160 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-ac89e828-3099-48c7-a2b7-e7103084765d None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:57.167908 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c64207-1b4d-4923-951b-161a018e6738 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:57.168748 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c64207-1b4d-4923-951b-161a018e6738 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:57.169260 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-ac89e828-3099-48c7-a2b7-e7103084765d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:57.171822 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c64207-1b4d-4923-951b-161a018e6738 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:57.212422 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 370/734] 10.222.0.55 () {56 vars in 1135 bytes} [Wed Aug 24 08:34:56 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:57.213407 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c64207-1b4d-4923-951b-161a018e6738 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:57.277201 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:57.291047 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:57.311854 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 365/735] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:34:57.313847 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac89e828-3099-48c7-a2b7-e7103084765d None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:57.403354 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c64207-1b4d-4923-951b-161a018e6738 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:57.468009 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660fce1b-5b72-4ff4-9ad9-3adf77431525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:57.468948 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660fce1b-5b72-4ff4-9ad9-3adf77431525 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:57.469423 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660fce1b-5b72-4ff4-9ad9-3adf77431525 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:57.469901 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660fce1b-5b72-4ff4-9ad9-3adf77431525 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:57.488372 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660fce1b-5b72-4ff4-9ad9-3adf77431525 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:57.491058 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660fce1b-5b72-4ff4-9ad9-3adf77431525 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:57.522719 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-660fce1b-5b72-4ff4-9ad9-3adf77431525 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:34:57.522719 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:57.522719 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:57.522719 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:57.522719 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:57.522719 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:57.522719 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:57.522719 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:57.522719 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:57.522719 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:57.522719 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:57.522719 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:57.522719 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:34:57.522719 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:34:57.522719 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:34:57.522719 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:34:57.524656 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:57.524656 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:57.524656 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:57.524656 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:57.524656 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:57.524656 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:57.524656 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:57.524656 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:57.524656 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:57.524656 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:57.524656 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:57.524656 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:57.524656 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:57.524656 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:57.524656 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:57.524656 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:57.524656 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:34:57.526870 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:34:57.526870 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:34:57.526870 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:34:57.526870 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 371/736] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:34:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:57.526870 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660fce1b-5b72-4ff4-9ad9-3adf77431525 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:57.562292 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac89e828-3099-48c7-a2b7-e7103084765d None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:57.623516 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c06ac9-dfe3-4cdf-8bbd-de24a3acb08e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:57.625110 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c06ac9-dfe3-4cdf-8bbd-de24a3acb08e None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:57.625841 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c06ac9-dfe3-4cdf-8bbd-de24a3acb08e None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:57.626538 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c06ac9-dfe3-4cdf-8bbd-de24a3acb08e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:57.719222 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:57.730245 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:57.733242 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660fce1b-5b72-4ff4-9ad9-3adf77431525 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:57.738237 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c06ac9-dfe3-4cdf-8bbd-de24a3acb08e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:57.739790 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c06ac9-dfe3-4cdf-8bbd-de24a3acb08e None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:57.776286 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e254c71-b941-4d25-8d92-0f18327c116b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:57.777102 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e254c71-b941-4d25-8d92-0f18327c116b None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:57.777550 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e254c71-b941-4d25-8d92-0f18327c116b None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:57.777902 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e254c71-b941-4d25-8d92-0f18327c116b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:57.791142 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e254c71-b941-4d25-8d92-0f18327c116b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:34:57.797398 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-7e254c71-b941-4d25-8d92-0f18327c116b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:34:57.797398 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:57.797398 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:57.797398 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:57.797398 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:57.797398 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:57.797398 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:57.797398 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:57.797398 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:57.797398 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:57.797398 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:57.797398 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:57.797398 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:34:57.797398 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:34:57.797398 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:34:57.797398 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:34:57.799487 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:34:57.799487 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:34:57.799487 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:34:57.799487 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:34:57.799487 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:57.799487 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:57.799487 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:57.799487 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:57.799487 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:57.799487 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:57.799487 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:57.799487 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:57.799487 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:57.799487 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:57.799487 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:57.799487 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:57.801404 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:57.801404 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:57.801404 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:57.801404 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:57.801404 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:34:57.801404 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:34:57.801404 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:34:57.801404 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:34:57.801404 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:34:57.801404 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:34:57.801404 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:34:57.801404 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:34:57.801404 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 372/737] 10.222.0.55 () {56 vars in 1096 bytes} [Wed Aug 24 08:34:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:57.801404 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e254c71-b941-4d25-8d92-0f18327c116b None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:57.870029 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 366/738] 10.222.0.55 () {58 vars in 1282 bytes} [Wed Aug 24 08:34:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 557 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:34:57.870870 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c06ac9-dfe3-4cdf-8bbd-de24a3acb08e None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:57.917475 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e254c71-b941-4d25-8d92-0f18327c116b None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:57.955493 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48af63e7-dbf6-41cc-91fd-9aeecb3e2521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:57.956109 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48af63e7-dbf6-41cc-91fd-9aeecb3e2521 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:57.956390 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48af63e7-dbf6-41cc-91fd-9aeecb3e2521 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:57.956688 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48af63e7-dbf6-41cc-91fd-9aeecb3e2521 None admin] PATH_INFO: `/v3/domains` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:57.957343 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48af63e7-dbf6-41cc-91fd-9aeecb3e2521 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:57.957646 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48af63e7-dbf6-41cc-91fd-9aeecb3e2521 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:57.959022 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48af63e7-dbf6-41cc-91fd-9aeecb3e2521 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:57.971878 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 373/739] 10.222.0.55 () {56 vars in 1109 bytes} [Wed Aug 24 08:34:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:57.973096 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f988d5-0025-441c-9576-5a8bb203b638 None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:57.973377 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f988d5-0025-441c-9576-5a8bb203b638 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:57.973647 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f988d5-0025-441c-9576-5a8bb203b638 None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:57.977818 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 374/740] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:34:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:34:57.977818 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f988d5-0025-441c-9576-5a8bb203b638 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:57.992423 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:58.013520 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:58.021365 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c06ac9-dfe3-4cdf-8bbd-de24a3acb08e None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:58.061756 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a67753-b546-47cb-aac2-dc746bf6668f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:58.062792 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a67753-b546-47cb-aac2-dc746bf6668f None admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:58.063146 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a67753-b546-47cb-aac2-dc746bf6668f None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:58.063511 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a67753-b546-47cb-aac2-dc746bf6668f None admin] PATH_INFO: `/v3/services` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:58.064628 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a67753-b546-47cb-aac2-dc746bf6668f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:58.066363 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a67753-b546-47cb-aac2-dc746bf6668f None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:58.108613 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f988d5-0025-441c-9576-5a8bb203b638 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:58.144907 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 367/741] 10.222.0.55 () {60 vars in 1141 bytes} [Wed Aug 24 08:34:57 2022] POST /identity/v3/services => generated 240 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:34:58.146384 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a67753-b546-47cb-aac2-dc746bf6668f None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:58.159435 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bebea0-e786-4bee-9e19-2e0ff5d4393b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:58.160258 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bebea0-e786-4bee-9e19-2e0ff5d4393b None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:58.160771 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bebea0-e786-4bee-9e19-2e0ff5d4393b None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:58.161303 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bebea0-e786-4bee-9e19-2e0ff5d4393b None admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:58.162451 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bebea0-e786-4bee-9e19-2e0ff5d4393b None admin] RBAC: Adding query filter params (name=admin) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:58.162949 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bebea0-e786-4bee-9e19-2e0ff5d4393b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:58.164613 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bebea0-e786-4bee-9e19-2e0ff5d4393b None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:58.178374 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 375/742] 10.222.0.55 () {56 vars in 1099 bytes} [Wed Aug 24 08:34:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:58.179818 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bebea0-e786-4bee-9e19-2e0ff5d4393b None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:58.284569 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:58.338500 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bebea0-e786-4bee-9e19-2e0ff5d4393b None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:58.365969 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a67753-b546-47cb-aac2-dc746bf6668f None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:58.389248 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efd62f3-a09d-4bb6-8631-20c45283f45d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:58.389905 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efd62f3-a09d-4bb6-8631-20c45283f45d None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:58.390248 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efd62f3-a09d-4bb6-8631-20c45283f45d None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:58.390523 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efd62f3-a09d-4bb6-8631-20c45283f45d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:58.400391 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efd62f3-a09d-4bb6-8631-20c45283f45d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:58.401906 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efd62f3-a09d-4bb6-8631-20c45283f45d None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:58.414619 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-4efd62f3-a09d-4bb6-8631-20c45283f45d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 08:34:58.414619 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:58.414619 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:58.414619 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:58.414619 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:58.414619 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:58.414619 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:58.414619 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:58.414619 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:58.414619 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:58.414619 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:58.414619 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:58.414619 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:34:58.414619 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:34:58.414619 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:34:58.414619 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:34:58.416517 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:58.416517 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:58.416517 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:34:58.416517 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:34:58.416517 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:34:58.416517 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:34:58.416517 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:58.416517 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:58.416517 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:58.416517 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:58.416517 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:58.416517 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:58.416517 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:58.416517 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:58.416517 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:58.416517 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:58.416517 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:58.416517 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:58.419847 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:58.419847 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:58.419847 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:34:58.419847 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:34:58.419847 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:34:58.419847 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:34:58.419847 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:34:58.419847 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:34:58.419847 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 08:34:58.419847 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:34:58.419847 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 376/743] 10.222.0.55 () {56 vars in 1117 bytes} [Wed Aug 24 08:34:58 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:58.419847 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cec8e60-706e-40c4-b028-d36a4ca980e7 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:58.419847 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cec8e60-706e-40c4-b028-d36a4ca980e7 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:58.422609 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cec8e60-706e-40c4-b028-d36a4ca980e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:58.433328 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c5d963-10c8-4b4e-8e1c-764ebe5fded3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:58.435085 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c5d963-10c8-4b4e-8e1c-764ebe5fded3 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:58.435411 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c5d963-10c8-4b4e-8e1c-764ebe5fded3 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:58.435813 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c5d963-10c8-4b4e-8e1c-764ebe5fded3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:58.439285 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-8cec8e60-706e-40c4-b028-d36a4ca980e7 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:34:58.440681 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-8cec8e60-706e-40c4-b028-d36a4ca980e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:34:58.455287 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c5d963-10c8-4b4e-8e1c-764ebe5fded3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:58.458039 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c5d963-10c8-4b4e-8e1c-764ebe5fded3 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:58.486941 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-c2c5d963-10c8-4b4e-8e1c-764ebe5fded3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:34:58.486941 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:58.486941 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:58.486941 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:58.486941 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:58.486941 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:58.486941 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:58.486941 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:58.486941 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:58.486941 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:58.486941 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:58.486941 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:58.486941 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:34:58.486941 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:34:58.486941 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:34:58.486941 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:34:58.489667 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:58.489667 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:58.489667 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:58.489667 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:58.489667 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:58.489667 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:58.489667 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:58.489667 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:58.489667 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:58.489667 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:58.489667 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:58.489667 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:58.489667 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:58.489667 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:58.489667 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:58.489667 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:58.489667 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:34:58.493757 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:34:58.493757 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:34:58.493757 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:34:58.493757 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 368/744] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:34:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:58.493757 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c5d963-10c8-4b4e-8e1c-764ebe5fded3 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:58.508623 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:58.552681 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 377/745] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:34:58 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:34:58.552681 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cec8e60-706e-40c4-b028-d36a4ca980e7 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:58.623980 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:58.683197 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cec8e60-706e-40c4-b028-d36a4ca980e7 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:58.684751 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c5d963-10c8-4b4e-8e1c-764ebe5fded3 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:58.733563 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3098b0a-d67f-4471-a8b1-a34f2ff2914c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:58.734627 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3098b0a-d67f-4471-a8b1-a34f2ff2914c None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:58.735154 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3098b0a-d67f-4471-a8b1-a34f2ff2914c None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:58.735566 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3098b0a-d67f-4471-a8b1-a34f2ff2914c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:58.736657 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3129f7ca-5374-47d9-96aa-d1df09707c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:58.737587 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3129f7ca-5374-47d9-96aa-d1df09707c38 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:58.737964 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3129f7ca-5374-47d9-96aa-d1df09707c38 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:58.738337 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3129f7ca-5374-47d9-96aa-d1df09707c38 None admin] PATH_INFO: `/v3/users` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:58.739527 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3129f7ca-5374-47d9-96aa-d1df09707c38 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:58.740198 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3129f7ca-5374-47d9-96aa-d1df09707c38 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:58.742539 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3129f7ca-5374-47d9-96aa-d1df09707c38 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:58.749467 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3098b0a-d67f-4471-a8b1-a34f2ff2914c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:58.751160 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3098b0a-d67f-4471-a8b1-a34f2ff2914c None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:58.767213 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-c3098b0a-d67f-4471-a8b1-a34f2ff2914c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:34:58.767213 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:58.767213 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:58.767213 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:58.767213 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:58.767213 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:58.767213 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:58.767213 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:58.767213 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:58.767213 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:58.767213 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:58.767213 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:58.767213 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:34:58.767213 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:34:58.767213 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:34:58.767213 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:34:58.768331 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:58.768331 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:58.768331 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:58.768331 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:58.768331 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:58.768331 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:58.768331 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:58.768331 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:58.768331 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:58.768331 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:58.768331 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:58.768331 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:58.768331 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:58.768331 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:58.768331 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:58.768331 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:58.768331 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:34:58.769259 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:34:58.769259 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:34:58.769259 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:34:58.771078 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 369/746] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:34:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:58.772090 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3098b0a-d67f-4471-a8b1-a34f2ff2914c None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:58.774690 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 378/747] 10.222.0.55 () {56 vars in 1149 bytes} [Wed Aug 24 08:34:58 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:58.781229 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3129f7ca-5374-47d9-96aa-d1df09707c38 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:58.861774 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:58.917426 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3098b0a-d67f-4471-a8b1-a34f2ff2914c None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:58.952578 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3129f7ca-5374-47d9-96aa-d1df09707c38 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:58.959269 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168acff6-ade2-4395-9f30-e1f8d8b7a3ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:58.959883 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168acff6-ade2-4395-9f30-e1f8d8b7a3ad None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:58.960168 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168acff6-ade2-4395-9f30-e1f8d8b7a3ad None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:58.960407 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168acff6-ade2-4395-9f30-e1f8d8b7a3ad None admin] PATH_INFO: `/v3/domains` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:58.961091 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168acff6-ade2-4395-9f30-e1f8d8b7a3ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:58.961390 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168acff6-ade2-4395-9f30-e1f8d8b7a3ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:58.962756 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168acff6-ade2-4395-9f30-e1f8d8b7a3ad None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:58.976840 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:58.980984 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 370/748] 10.222.0.55 () {56 vars in 1109 bytes} [Wed Aug 24 08:34:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:58.981789 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168acff6-ade2-4395-9f30-e1f8d8b7a3ad None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:59.007418 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ead106-7d55-4b95-b5a2-3ab0f1bc0192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:59.008353 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ead106-7d55-4b95-b5a2-3ab0f1bc0192 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:59.008881 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ead106-7d55-4b95-b5a2-3ab0f1bc0192 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:59.010276 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ead106-7d55-4b95-b5a2-3ab0f1bc0192 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:59.107119 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ead106-7d55-4b95-b5a2-3ab0f1bc0192 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:59.109105 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ead106-7d55-4b95-b5a2-3ab0f1bc0192 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:59.146326 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168acff6-ade2-4395-9f30-e1f8d8b7a3ad None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:59.170873 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:59.205650 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d7efdb-62ce-4358-9c95-54298e686c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:59.206718 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d7efdb-62ce-4358-9c95-54298e686c06 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:59.207218 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d7efdb-62ce-4358-9c95-54298e686c06 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:59.207483 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d7efdb-62ce-4358-9c95-54298e686c06 None admin] PATH_INFO: `/v3/domains` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:59.215206 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d7efdb-62ce-4358-9c95-54298e686c06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:34:59.215206 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d7efdb-62ce-4358-9c95-54298e686c06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:59.215206 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d7efdb-62ce-4358-9c95-54298e686c06 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:59.242505 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 371/749] 10.222.0.55 () {56 vars in 1109 bytes} [Wed Aug 24 08:34:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:34:59.244518 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d7efdb-62ce-4358-9c95-54298e686c06 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:59.261073 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 379/750] 10.222.0.55 () {58 vars in 1282 bytes} [Wed Aug 24 08:34:58 2022] GET /identity/v3/auth/tokens => generated 898 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:34:59.261073 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ead106-7d55-4b95-b5a2-3ab0f1bc0192 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:59.285784 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:59.399098 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d7efdb-62ce-4358-9c95-54298e686c06 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:59.420570 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ead106-7d55-4b95-b5a2-3ab0f1bc0192 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:59.444674 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6c2a34-aee0-4842-bb9c-98f5fa7b5093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:59.445266 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6c2a34-aee0-4842-bb9c-98f5fa7b5093 None admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:59.445541 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6c2a34-aee0-4842-bb9c-98f5fa7b5093 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:59.445815 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6c2a34-aee0-4842-bb9c-98f5fa7b5093 None admin] PATH_INFO: `/v3/services` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:59.446462 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6c2a34-aee0-4842-bb9c-98f5fa7b5093 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:59.447810 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6c2a34-aee0-4842-bb9c-98f5fa7b5093 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:59.461368 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ec2505-4107-4584-a637-5000a7946b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:59.462022 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ec2505-4107-4584-a637-5000a7946b0d None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:59.465955 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ec2505-4107-4584-a637-5000a7946b0d None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:59.465955 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ec2505-4107-4584-a637-5000a7946b0d None admin] PATH_INFO: `/v3/projects/service` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:59.469330 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ec2505-4107-4584-a637-5000a7946b0d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:59.470742 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ec2505-4107-4584-a637-5000a7946b0d None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:59.482463 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-89ec2505-4107-4584-a637-5000a7946b0d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:34:59.482463 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:59.482463 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:59.482463 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:59.482463 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:59.482463 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:59.482463 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:59.482463 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:59.482463 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:59.482463 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:59.482463 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:59.482463 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:59.482463 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:34:59.482463 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:34:59.482463 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:34:59.483393 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:34:59.483393 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:59.483393 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:59.483393 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:59.483393 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:59.483393 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:59.483393 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:59.483393 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:59.483393 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:59.483393 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:59.483393 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:59.483393 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:59.483393 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:59.483393 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:59.483393 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:59.483393 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:59.483393 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:59.485456 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:34:59.485456 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:34:59.485456 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:34:59.485456 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:34:59.485456 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:34:59.485456 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:34:59.485456 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:34:59.485456 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:34:59.485456 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 380/751] 10.222.0.55 () {56 vars in 1111 bytes} [Wed Aug 24 08:34:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:59.485456 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ec2505-4107-4584-a637-5000a7946b0d None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:59.546887 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 372/752] 10.222.0.55 () {60 vars in 1141 bytes} [Wed Aug 24 08:34:59 2022] POST /identity/v3/services => generated 251 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:34:59.548203 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6c2a34-aee0-4842-bb9c-98f5fa7b5093 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:59.631260 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:59.650714 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:34:59.685318 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ec2505-4107-4584-a637-5000a7946b0d None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:59.770816 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2655d5-5f44-486c-bd2d-ed1a11af7a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:59.771885 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2655d5-5f44-486c-bd2d-ed1a11af7a9c None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:59.772334 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2655d5-5f44-486c-bd2d-ed1a11af7a9c None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:59.772831 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2655d5-5f44-486c-bd2d-ed1a11af7a9c None admin] PATH_INFO: `/v3/users/nova` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:59.787935 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2655d5-5f44-486c-bd2d-ed1a11af7a9c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:59.789588 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6c2a34-aee0-4842-bb9c-98f5fa7b5093 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:34:59.790638 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2655d5-5f44-486c-bd2d-ed1a11af7a9c None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:59.814013 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-9d2655d5-5f44-486c-bd2d-ed1a11af7a9c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 08:34:59.814013 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:59.814013 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:59.814013 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:59.814013 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:59.814013 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:59.814013 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:59.814013 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:59.814013 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:59.814013 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:59.814013 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:59.814013 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:59.814013 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:34:59.814013 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:34:59.814013 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:34:59.814013 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:34:59.815053 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:59.815053 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:59.815053 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:34:59.815053 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:34:59.815053 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:34:59.815053 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:34:59.815053 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:59.815053 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:59.815053 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:59.815053 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:59.815053 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:59.815053 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:59.815053 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:59.815053 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:59.815053 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:59.815053 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:59.815053 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:59.815053 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:59.816054 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:59.816054 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:59.816054 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:34:59.816054 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:34:59.816054 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:34:59.816054 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:34:59.816054 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:34:59.816054 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:34:59.816054 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 08:34:59.816054 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:34:59.816054 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 381/753] 10.222.0.55 () {56 vars in 1093 bytes} [Wed Aug 24 08:34:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:59.817233 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2655d5-5f44-486c-bd2d-ed1a11af7a9c None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:59.857032 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad067dd-2c45-449f-9a67-3030f028850b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:34:59.858015 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad067dd-2c45-449f-9a67-3030f028850b None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:34:59.858499 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad067dd-2c45-449f-9a67-3030f028850b None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:34:59.858801 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad067dd-2c45-449f-9a67-3030f028850b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:34:59.878004 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad067dd-2c45-449f-9a67-3030f028850b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:34:59.880500 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad067dd-2c45-449f-9a67-3030f028850b None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:34:59.905806 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-dad067dd-2c45-449f-9a67-3030f028850b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:34:59.905806 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:34:59.905806 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:34:59.905806 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:34:59.905806 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:34:59.905806 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:34:59.905806 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:34:59.905806 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:34:59.905806 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:34:59.905806 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:34:59.905806 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:34:59.905806 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:34:59.905806 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:34:59.905806 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:34:59.905806 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:34:59.905806 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:34:59.906819 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:34:59.906819 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:34:59.906819 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:34:59.906819 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:34:59.906819 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:34:59.906819 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:34:59.906819 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:34:59.906819 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:34:59.906819 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:34:59.906819 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:34:59.906819 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:34:59.906819 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:34:59.906819 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:34:59.906819 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:34:59.906819 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:34:59.906819 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:34:59.906819 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:34:59.907748 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:34:59.907748 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:34:59.907748 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:34:59.908379 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 373/754] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:34:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 360 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:34:59.910425 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad067dd-2c45-449f-9a67-3030f028850b None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:34:59.975137 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2655d5-5f44-486c-bd2d-ed1a11af7a9c None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:00.010265 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:00.018661 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2610b8-ed66-4573-84e3-5ce56e9a3541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:00.019775 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2610b8-ed66-4573-84e3-5ce56e9a3541 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:00.020356 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2610b8-ed66-4573-84e3-5ce56e9a3541 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:00.020939 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2610b8-ed66-4573-84e3-5ce56e9a3541 None admin] PATH_INFO: `/v3/projects` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:00.022315 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2610b8-ed66-4573-84e3-5ce56e9a3541 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:00.022975 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2610b8-ed66-4573-84e3-5ce56e9a3541 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:00.025644 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2610b8-ed66-4573-84e3-5ce56e9a3541 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:00.057726 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 382/755] 10.222.0.55 () {56 vars in 1148 bytes} [Wed Aug 24 08:34:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:00.059823 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2610b8-ed66-4573-84e3-5ce56e9a3541 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:00.067763 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:00.081087 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad067dd-2c45-449f-9a67-3030f028850b None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:00.146959 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8931f2e-26d2-4d35-9a4b-47cb51aed399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:00.147992 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8931f2e-26d2-4d35-9a4b-47cb51aed399 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:00.148433 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8931f2e-26d2-4d35-9a4b-47cb51aed399 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:00.148934 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8931f2e-26d2-4d35-9a4b-47cb51aed399 None admin] PATH_INFO: `/v3/users` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:00.150338 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8931f2e-26d2-4d35-9a4b-47cb51aed399 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:00.150910 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8931f2e-26d2-4d35-9a4b-47cb51aed399 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:00.154286 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8931f2e-26d2-4d35-9a4b-47cb51aed399 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:00.188901 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 374/756] 10.222.0.55 () {56 vars in 1133 bytes} [Wed Aug 24 08:34:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:00.190221 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8931f2e-26d2-4d35-9a4b-47cb51aed399 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:00.241189 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2610b8-ed66-4573-84e3-5ce56e9a3541 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:00.298578 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff57559-8bca-48f2-b58b-6ba00f503016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:00.299194 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff57559-8bca-48f2-b58b-6ba00f503016 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:00.299465 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff57559-8bca-48f2-b58b-6ba00f503016 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:00.299737 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff57559-8bca-48f2-b58b-6ba00f503016 None admin] PATH_INFO: `/v3/domains` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:00.300397 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff57559-8bca-48f2-b58b-6ba00f503016 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:00.300729 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff57559-8bca-48f2-b58b-6ba00f503016 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:00.302102 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff57559-8bca-48f2-b58b-6ba00f503016 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:00.314543 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 383/757] 10.222.0.55 () {56 vars in 1109 bytes} [Wed Aug 24 08:35:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:00.315820 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ab02ac-2fb3-429e-91e2-e0691dd78f50 None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:00.316102 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ab02ac-2fb3-429e-91e2-e0691dd78f50 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:00.316375 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ab02ac-2fb3-429e-91e2-e0691dd78f50 None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:00.320592 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 384/758] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:35:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:35:00.320592 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ab02ac-2fb3-429e-91e2-e0691dd78f50 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:00.361525 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8931f2e-26d2-4d35-9a4b-47cb51aed399 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:00.375073 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:00.425765 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af4f92a-3548-441f-a276-9c98dee089d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:00.426602 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af4f92a-3548-441f-a276-9c98dee089d3 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:00.426998 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af4f92a-3548-441f-a276-9c98dee089d3 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:00.427407 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af4f92a-3548-441f-a276-9c98dee089d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:00.428478 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af4f92a-3548-441f-a276-9c98dee089d3 None admin] RBAC: Adding query filter params (role.id=b870cd1acaa744e1ae1d58b9ac283821, scope.project.id=420b18725f2342159646fb86438318fd, user.id=a0ebb5ccabef45378f0c1da69721e051) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:00.428972 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af4f92a-3548-441f-a276-9c98dee089d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:00.431110 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af4f92a-3548-441f-a276-9c98dee089d3 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:00.441160 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 375/759] 10.222.0.55 () {56 vars in 1374 bytes} [Wed Aug 24 08:35:00 2022] GET /identity/v3/role_assignments?user.id=a0ebb5ccabef45378f0c1da69721e051&scope.project.id=420b18725f2342159646fb86438318fd&role.id=b870cd1acaa744e1ae1d58b9ac283821 => generated 259 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:00.441951 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ab02ac-2fb3-429e-91e2-e0691dd78f50 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:00.442503 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af4f92a-3548-441f-a276-9c98dee089d3 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:00.478694 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:00.517684 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd41ef9-b1a1-4fb0-ab54-3c1cf4d47292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:00.518990 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd41ef9-b1a1-4fb0-ab54-3c1cf4d47292 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:00.519678 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd41ef9-b1a1-4fb0-ab54-3c1cf4d47292 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:00.520394 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd41ef9-b1a1-4fb0-ab54-3c1cf4d47292 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:00.542003 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd41ef9-b1a1-4fb0-ab54-3c1cf4d47292 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:00.544980 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd41ef9-b1a1-4fb0-ab54-3c1cf4d47292 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:00.572255 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-9fd41ef9-b1a1-4fb0-ab54-3c1cf4d47292 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:00.572255 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:00.572255 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:00.572255 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:00.572255 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:00.572255 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:00.572255 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:00.572255 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:00.572255 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:00.572255 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:00.572255 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:00.572255 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:00.572255 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:35:00.572255 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:35:00.572255 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:35:00.572255 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:35:00.574276 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:00.574276 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:00.574276 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:00.574276 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:00.574276 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:00.574276 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:00.574276 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:00.574276 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:00.574276 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:00.574276 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:00.574276 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:00.574276 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:00.574276 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:00.574276 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:00.574276 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:00.574276 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:00.574276 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:35:00.576241 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:35:00.576241 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:00.576241 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:35:00.579672 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 385/760] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:35:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:00.582763 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ec2206-39f9-43db-b07b-768620b1804f None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:00.583465 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ec2206-39f9-43db-b07b-768620b1804f None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:00.584142 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ec2206-39f9-43db-b07b-768620b1804f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:00.625879 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-d8ec2206-39f9-43db-b07b-768620b1804f None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:35:00.628240 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-d8ec2206-39f9-43db-b07b-768620b1804f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:35:00.669587 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af4f92a-3548-441f-a276-9c98dee089d3 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:00.736033 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451b18a3-4639-4a98-80c4-e1c9ac29c3b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:00.736633 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451b18a3-4639-4a98-80c4-e1c9ac29c3b1 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:00.737054 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451b18a3-4639-4a98-80c4-e1c9ac29c3b1 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:00.737331 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451b18a3-4639-4a98-80c4-e1c9ac29c3b1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:00.751392 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451b18a3-4639-4a98-80c4-e1c9ac29c3b1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:00.754496 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451b18a3-4639-4a98-80c4-e1c9ac29c3b1 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:00.765032 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 386/761] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:35:00 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:35:00.766949 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ec2206-39f9-43db-b07b-768620b1804f None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:00.782106 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-451b18a3-4639-4a98-80c4-e1c9ac29c3b1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:35:00.782106 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:00.782106 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:00.782106 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:00.782106 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:00.782106 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:00.782106 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:00.782106 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:00.782106 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:00.782106 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:00.782106 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:00.782106 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:00.782106 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:35:00.782106 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:35:00.782106 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:35:00.783824 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:35:00.783824 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:00.783824 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:00.783824 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:00.783824 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:00.783824 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:00.783824 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:00.783824 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:00.783824 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:00.783824 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:00.783824 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:00.783824 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:00.783824 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:00.783824 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:00.783824 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:00.783824 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:00.783824 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:00.785506 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:35:00.785506 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:35:00.785506 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:35:00.785506 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:35:00.785506 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:35:00.785506 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:35:00.785506 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:35:00.785506 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:35:00.792477 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 376/762] 10.222.0.55 () {56 vars in 1111 bytes} [Wed Aug 24 08:35:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:00.795023 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451b18a3-4639-4a98-80c4-e1c9ac29c3b1 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:00.845041 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:00.973168 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:01.008592 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451b18a3-4639-4a98-80c4-e1c9ac29c3b1 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:01.020134 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ec2206-39f9-43db-b07b-768620b1804f None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:01.078080 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b41453-831f-4a6e-8ed0-827446f563a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:01.079019 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b41453-831f-4a6e-8ed0-827446f563a5 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:01.079464 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b41453-831f-4a6e-8ed0-827446f563a5 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:01.079910 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b41453-831f-4a6e-8ed0-827446f563a5 None admin] PATH_INFO: `/v3/projects` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:01.081180 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b41453-831f-4a6e-8ed0-827446f563a5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:01.081679 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b41453-831f-4a6e-8ed0-827446f563a5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:01.084164 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b41453-831f-4a6e-8ed0-827446f563a5 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:01.126153 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 377/763] 10.222.0.55 () {56 vars in 1148 bytes} [Wed Aug 24 08:35:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:01.131682 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e03c0d2-5164-4551-88ed-800b58caa2ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:01.132738 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e03c0d2-5164-4551-88ed-800b58caa2ff None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:01.133229 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e03c0d2-5164-4551-88ed-800b58caa2ff None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:01.133714 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e03c0d2-5164-4551-88ed-800b58caa2ff None admin] PATH_INFO: `/v3/domains` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:01.134910 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e03c0d2-5164-4551-88ed-800b58caa2ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:01.137113 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b41453-831f-4a6e-8ed0-827446f563a5 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:01.137793 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e03c0d2-5164-4551-88ed-800b58caa2ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:01.140477 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e03c0d2-5164-4551-88ed-800b58caa2ff None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:01.180280 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 387/764] 10.222.0.55 () {56 vars in 1109 bytes} [Wed Aug 24 08:35:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 414 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:01.195303 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e03c0d2-5164-4551-88ed-800b58caa2ff None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:01.363704 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b41453-831f-4a6e-8ed0-827446f563a5 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:01.413324 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:01.420964 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:01.429907 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d11cb3d-06d5-4189-9474-7ccdb823f726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:01.432163 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d11cb3d-06d5-4189-9474-7ccdb823f726 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:01.432163 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d11cb3d-06d5-4189-9474-7ccdb823f726 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:01.432163 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d11cb3d-06d5-4189-9474-7ccdb823f726 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:01.432639 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e03c0d2-5164-4551-88ed-800b58caa2ff None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:01.434047 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d11cb3d-06d5-4189-9474-7ccdb823f726 None admin] RBAC: Adding query filter params (role.id=bab90e02eebc473e9b911c8660ed9b76, scope.project.id=420b18725f2342159646fb86438318fd, user.id=f652a368b1cc42ae8b6f6c7a6dff6f2f) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:01.435359 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d11cb3d-06d5-4189-9474-7ccdb823f726 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:01.439961 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d11cb3d-06d5-4189-9474-7ccdb823f726 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:01.456388 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 378/765] 10.222.0.55 () {56 vars in 1374 bytes} [Wed Aug 24 08:35:01 2022] GET /identity/v3/role_assignments?user.id=f652a368b1cc42ae8b6f6c7a6dff6f2f&scope.project.id=420b18725f2342159646fb86438318fd&role.id=bab90e02eebc473e9b911c8660ed9b76 => generated 259 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:35:01.461114 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d11cb3d-06d5-4189-9474-7ccdb823f726 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:01.555466 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a02250e-8728-4c78-94f0-506b48ba1362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:01.556517 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a02250e-8728-4c78-94f0-506b48ba1362 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:01.557042 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a02250e-8728-4c78-94f0-506b48ba1362 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:01.557701 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a02250e-8728-4c78-94f0-506b48ba1362 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:01.572123 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a02250e-8728-4c78-94f0-506b48ba1362 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:01.574468 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a02250e-8728-4c78-94f0-506b48ba1362 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:01.594224 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-1a02250e-8728-4c78-94f0-506b48ba1362 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:35:01.594224 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:01.594224 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:01.594224 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:01.594224 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:01.594224 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:01.594224 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:01.594224 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:01.594224 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:01.594224 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:01.594224 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:01.594224 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:01.594224 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:35:01.594224 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:35:01.594224 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:35:01.596192 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:35:01.596192 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:01.596192 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:01.596192 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:01.596192 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:01.596192 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:01.596192 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:01.596192 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:01.596192 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:01.596192 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:01.596192 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:01.596192 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:01.596192 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:01.596192 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:01.596192 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:01.596192 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:01.596192 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:01.597947 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:35:01.597947 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:35:01.597947 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:35:01.597947 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:35:01.597947 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:35:01.597947 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:35:01.597947 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:35:01.597947 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:35:01.597947 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 388/766] 10.222.0.55 () {56 vars in 1111 bytes} [Wed Aug 24 08:35:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 402 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:01.605871 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a02250e-8728-4c78-94f0-506b48ba1362 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:01.703311 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d11cb3d-06d5-4189-9474-7ccdb823f726 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:01.776253 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de9ad08-8da1-44ad-9d2d-16e46acc519d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:01.777231 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de9ad08-8da1-44ad-9d2d-16e46acc519d None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:01.777778 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de9ad08-8da1-44ad-9d2d-16e46acc519d None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:01.778261 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de9ad08-8da1-44ad-9d2d-16e46acc519d None admin] PATH_INFO: `/v3/services/network` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:01.779069 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de9ad08-8da1-44ad-9d2d-16e46acc519d None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:01.781746 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de9ad08-8da1-44ad-9d2d-16e46acc519d None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:01.792689 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-4de9ad08-8da1-44ad-9d2d-16e46acc519d None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 08:35:01.792689 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:01.792689 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 08:35:01.792689 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 08:35:01.792689 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 08:35:01.792689 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 08:35:01.792689 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 08:35:01.792689 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:35:01.792689 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 08:35:01.792689 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:35:01.792689 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:35:01.792689 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:35:01.792689 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:01.792689 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:01.792689 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:01.792689 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:01.792689 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:01.793789 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:01.793789 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:01.793789 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:01.793789 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:01.793789 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:01.793789 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:01.793789 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 08:35:01.793789 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 08:35:01.793789 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 08:35:01.793789 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 08:35:01.793789 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:01.793789 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:01.793789 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:01.793789 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:01.793789 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:01.793789 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:01.794650 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:01.794650 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:01.794650 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:01.794650 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:01.794650 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:01.794650 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:01.794650 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:01.794650 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:01.794650 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:01.794650 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:01.794650 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 08:35:01.794650 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:35:01.794650 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 08:35:01.794650 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:35:01.797423 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 379/767] 10.222.0.55 () {56 vars in 1111 bytes} [Wed Aug 24 08:35:01 2022] GET /identity/v3/services/network => generated 88 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:01.803034 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de9ad08-8da1-44ad-9d2d-16e46acc519d None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:01.836704 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a02250e-8728-4c78-94f0-506b48ba1362 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:01.886753 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:01.929183 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de9ad08-8da1-44ad-9d2d-16e46acc519d None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:01.972636 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27da5351-4abc-4567-b7c5-3c6a512e7051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:01.973544 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27da5351-4abc-4567-b7c5-3c6a512e7051 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:01.973998 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27da5351-4abc-4567-b7c5-3c6a512e7051 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:01.974463 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27da5351-4abc-4567-b7c5-3c6a512e7051 None admin] PATH_INFO: `/v3/services` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:01.975216 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850619d3-eae5-4831-84ed-8da367003006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:01.975800 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27da5351-4abc-4567-b7c5-3c6a512e7051 None admin] RBAC: Adding query filter params (name=network) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:01.976273 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27da5351-4abc-4567-b7c5-3c6a512e7051 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:01.976439 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850619d3-eae5-4831-84ed-8da367003006 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:01.977096 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850619d3-eae5-4831-84ed-8da367003006 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:01.977667 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850619d3-eae5-4831-84ed-8da367003006 None admin] PATH_INFO: `/v3/projects` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:01.978441 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27da5351-4abc-4567-b7c5-3c6a512e7051 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:01.979108 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850619d3-eae5-4831-84ed-8da367003006 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:01.979724 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850619d3-eae5-4831-84ed-8da367003006 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:01.983391 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850619d3-eae5-4831-84ed-8da367003006 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:01.985311 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 380/768] 10.222.0.55 () {56 vars in 1112 bytes} [Wed Aug 24 08:35:01 2022] GET /identity/v3/services?name=network => generated 124 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:35:01.990650 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27da5351-4abc-4567-b7c5-3c6a512e7051 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:02.021852 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 389/769] 10.222.0.55 () {56 vars in 1148 bytes} [Wed Aug 24 08:35:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 417 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:02.027789 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850619d3-eae5-4831-84ed-8da367003006 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:02.060998 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:02.180691 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27da5351-4abc-4567-b7c5-3c6a512e7051 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:02.181994 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850619d3-eae5-4831-84ed-8da367003006 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:02.211798 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:02.229418 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1048fa32-5d21-4365-b98b-6ea2b782136e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:02.230055 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1048fa32-5d21-4365-b98b-6ea2b782136e None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:02.230403 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1048fa32-5d21-4365-b98b-6ea2b782136e None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:02.230745 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1048fa32-5d21-4365-b98b-6ea2b782136e None admin] PATH_INFO: `/v3/services` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:02.232112 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1048fa32-5d21-4365-b98b-6ea2b782136e None admin] RBAC: Adding query filter params (type=network) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:02.232482 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1048fa32-5d21-4365-b98b-6ea2b782136e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:02.233979 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1048fa32-5d21-4365-b98b-6ea2b782136e None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:02.234273 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15339ba-1f93-4aab-8fba-21182900e672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:02.235120 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15339ba-1f93-4aab-8fba-21182900e672 None admin] REQUEST_METHOD: `PUT` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:02.235548 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15339ba-1f93-4aab-8fba-21182900e672 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:02.236052 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15339ba-1f93-4aab-8fba-21182900e672 None admin] PATH_INFO: `/v3/projects/420b18725f2342159646fb86438318fd/users/f17da357e555420799ee681bffbcc268/roles/b870cd1acaa744e1ae1d58b9ac283821` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:02.238428 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15339ba-1f93-4aab-8fba-21182900e672 None admin] RBAC: Authorizing `identity:create_grant(project_id=420b18725f2342159646fb86438318fd, user_id=f17da357e555420799ee681bffbcc268, role_id=b870cd1acaa744e1ae1d58b9ac283821)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:02.241292 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 381/770] 10.222.0.55 () {56 vars in 1112 bytes} [Wed Aug 24 08:35:01 2022] GET /identity/v3/services?type=network => generated 350 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:35:02.241457 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15339ba-1f93-4aab-8fba-21182900e672 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:02.242570 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cba68b-2b40-475a-906c-7d63fc8ee61d None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:02.242856 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cba68b-2b40-475a-906c-7d63fc8ee61d None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:02.243185 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cba68b-2b40-475a-906c-7d63fc8ee61d None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:02.244769 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 382/771] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:35:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:35:02.250169 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa04e85-f9ac-4f71-a8c6-4a9bb570da20 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:02.250572 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa04e85-f9ac-4f71-a8c6-4a9bb570da20 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:02.250959 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa04e85-f9ac-4f71-a8c6-4a9bb570da20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:02.271437 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-caa04e85-f9ac-4f71-a8c6-4a9bb570da20 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:35:02.272817 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-caa04e85-f9ac-4f71-a8c6-4a9bb570da20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:35:02.335090 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 390/772] 10.222.0.55 () {58 vars in 1439 bytes} [Wed Aug 24 08:35:02 2022] PUT /identity/v3/projects/420b18725f2342159646fb86438318fd/users/f17da357e555420799ee681bffbcc268/roles/b870cd1acaa744e1ae1d58b9ac283821 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:35:02.337253 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15339ba-1f93-4aab-8fba-21182900e672 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:02.350606 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 383/773] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:35:02 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:35:02.467933 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:02.515840 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15339ba-1f93-4aab-8fba-21182900e672 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:02.586951 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131fcb14-c100-4317-a907-7aaa53ead44d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:02.588028 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131fcb14-c100-4317-a907-7aaa53ead44d None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:02.588512 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131fcb14-c100-4317-a907-7aaa53ead44d None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:02.589013 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131fcb14-c100-4317-a907-7aaa53ead44d None admin] PATH_INFO: `/v3/endpoints` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:02.590282 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131fcb14-c100-4317-a907-7aaa53ead44d None admin] RBAC: Adding query filter params (interface=public, service_id=fcdedd6d886242738036df7e17d44d1a, region_id=RegionOne) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:02.591442 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131fcb14-c100-4317-a907-7aaa53ead44d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:02.594035 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131fcb14-c100-4317-a907-7aaa53ead44d None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:02.611591 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 391/774] 10.222.0.55 () {56 vars in 1251 bytes} [Wed Aug 24 08:35:02 2022] GET /identity/v3/endpoints?service_id=fcdedd6d886242738036df7e17d44d1a&interface=public®ion_id=RegionOne => generated 194 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:02.617037 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa04e85-f9ac-4f71-a8c6-4a9bb570da20 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:02.768608 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa04e85-f9ac-4f71-a8c6-4a9bb570da20 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:02.787612 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:02.805678 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c11a804-b643-4b67-aa63-2be362cd19e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:02.806290 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c11a804-b643-4b67-aa63-2be362cd19e8 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:02.806565 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c11a804-b643-4b67-aa63-2be362cd19e8 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:02.806838 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c11a804-b643-4b67-aa63-2be362cd19e8 None admin] PATH_INFO: `/v3/services` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:02.807448 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c11a804-b643-4b67-aa63-2be362cd19e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:02.808906 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c11a804-b643-4b67-aa63-2be362cd19e8 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:02.815504 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 384/775] 10.222.0.55 () {56 vars in 1087 bytes} [Wed Aug 24 08:35:02 2022] GET /identity/v3/services => generated 772 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:02.898925 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131fcb14-c100-4317-a907-7aaa53ead44d None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:03.080336 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131fcb14-c100-4317-a907-7aaa53ead44d None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:03.121165 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b23f694-f2e5-46d4-ac3c-cac302707653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:03.121806 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b23f694-f2e5-46d4-ac3c-cac302707653 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:03.122127 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b23f694-f2e5-46d4-ac3c-cac302707653 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:03.122395 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b23f694-f2e5-46d4-ac3c-cac302707653 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:03.122980 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b23f694-f2e5-46d4-ac3c-cac302707653 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:03.124359 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b23f694-f2e5-46d4-ac3c-cac302707653 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:03.127450 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:03.134797 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-8b23f694-f2e5-46d4-ac3c-cac302707653 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 08:35:03.134797 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:03.134797 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 08:35:03.134797 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 08:35:03.134797 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 08:35:03.134797 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 08:35:03.134797 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 08:35:03.134797 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:35:03.134797 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 08:35:03.134797 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:35:03.134797 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:35:03.134797 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:35:03.134797 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:03.134797 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:03.134797 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:03.134797 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:03.134797 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:03.135840 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:03.135840 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:03.135840 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:03.135840 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:03.135840 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:03.135840 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:03.135840 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 08:35:03.135840 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 08:35:03.135840 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 08:35:03.135840 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 08:35:03.135840 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:03.135840 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:03.135840 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:03.135840 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:03.135840 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:03.135840 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:03.136726 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:03.136726 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:03.136726 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:03.136726 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:03.136726 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:03.136726 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:03.136726 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:03.136726 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:03.136726 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:03.136726 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:03.136726 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 08:35:03.136726 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:35:03.136726 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 08:35:03.136726 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:35:03.136726 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 392/776] 10.222.0.55 () {56 vars in 1129 bytes} [Wed Aug 24 08:35:02 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:03.159026 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c11a804-b643-4b67-aa63-2be362cd19e8 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:03.274886 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c11a804-b643-4b67-aa63-2be362cd19e8 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:03.277927 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b34cae-ace0-4cdd-b87a-fce402f272b4 None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:03.278750 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b34cae-ace0-4cdd-b87a-fce402f272b4 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:03.279035 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b34cae-ace0-4cdd-b87a-fce402f272b4 None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:03.280727 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 393/777] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:35:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:35:03.287535 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bf3c95-ae90-418e-ab90-e8e972184647 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:03.287839 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bf3c95-ae90-418e-ab90-e8e972184647 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:03.288088 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bf3c95-ae90-418e-ab90-e8e972184647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:03.309134 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-43bf3c95-ae90-418e-ab90-e8e972184647 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:35:03.310406 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-43bf3c95-ae90-418e-ab90-e8e972184647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:35:03.363503 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d13b089-1cbb-4c9a-9bfc-b343b309dbee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:03.364042 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d13b089-1cbb-4c9a-9bfc-b343b309dbee None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:03.364314 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d13b089-1cbb-4c9a-9bfc-b343b309dbee None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:03.364837 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d13b089-1cbb-4c9a-9bfc-b343b309dbee None admin] PATH_INFO: `/v3/services` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:03.365485 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d13b089-1cbb-4c9a-9bfc-b343b309dbee None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:03.365783 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d13b089-1cbb-4c9a-9bfc-b343b309dbee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:03.367173 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d13b089-1cbb-4c9a-9bfc-b343b309dbee None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:03.377492 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 385/778] 10.222.0.55 () {56 vars in 1124 bytes} [Wed Aug 24 08:35:03 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:03.387442 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d13b089-1cbb-4c9a-9bfc-b343b309dbee None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:03.454065 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 394/779] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:35:03 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 08:35:03.480986 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:03.607501 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d13b089-1cbb-4c9a-9bfc-b343b309dbee None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:03.683559 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c68481-47be-4482-96fa-43cd0fe3ef57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:03.685092 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c68481-47be-4482-96fa-43cd0fe3ef57 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:03.685553 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c68481-47be-4482-96fa-43cd0fe3ef57 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:03.686016 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c68481-47be-4482-96fa-43cd0fe3ef57 None admin] PATH_INFO: `/v3/services` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:03.687125 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c68481-47be-4482-96fa-43cd0fe3ef57 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:03.687702 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c68481-47be-4482-96fa-43cd0fe3ef57 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:03.690269 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c68481-47be-4482-96fa-43cd0fe3ef57 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:03.699633 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 386/780] 10.222.0.55 () {56 vars in 1124 bytes} [Wed Aug 24 08:35:03 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:03.709288 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bf3c95-ae90-418e-ab90-e8e972184647 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:03.896876 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bf3c95-ae90-418e-ab90-e8e972184647 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:03.964303 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:03.973218 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb52773-1321-4d56-aa92-8e4d977d2ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:03.974075 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb52773-1321-4d56-aa92-8e4d977d2ad7 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:03.974477 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb52773-1321-4d56-aa92-8e4d977d2ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:03.974869 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb52773-1321-4d56-aa92-8e4d977d2ad7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:03.976062 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb52773-1321-4d56-aa92-8e4d977d2ad7 None admin] RBAC: Adding query filter params (interface=public, service_id=dcc810f6afd2463bbdbd55134464c78b, region_id=RegionOne) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:03.976587 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb52773-1321-4d56-aa92-8e4d977d2ad7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:03.978690 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb52773-1321-4d56-aa92-8e4d977d2ad7 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:03.992257 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 395/781] 10.222.0.55 () {56 vars in 1251 bytes} [Wed Aug 24 08:35:03 2022] GET /identity/v3/endpoints?service_id=dcc810f6afd2463bbdbd55134464c78b&interface=public®ion_id=RegionOne => generated 194 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:03.995736 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c68481-47be-4482-96fa-43cd0fe3ef57 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:04.013452 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb52773-1321-4d56-aa92-8e4d977d2ad7 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:04.172591 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c68481-47be-4482-96fa-43cd0fe3ef57 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:04.187429 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:04.214028 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb52773-1321-4d56-aa92-8e4d977d2ad7 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:04.222057 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86493ef-f894-4c6a-b34e-ac99af4a86bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:04.223506 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86493ef-f894-4c6a-b34e-ac99af4a86bf None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:04.223972 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86493ef-f894-4c6a-b34e-ac99af4a86bf None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:04.224435 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86493ef-f894-4c6a-b34e-ac99af4a86bf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:04.239338 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86493ef-f894-4c6a-b34e-ac99af4a86bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:35:04.246001 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-b86493ef-f894-4c6a-b34e-ac99af4a86bf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:35:04.246001 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:04.246001 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:04.246001 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:04.246001 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:04.246001 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:04.246001 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:04.246001 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:04.246001 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:04.246001 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:04.246001 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:04.246001 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:04.246001 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:35:04.246001 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:35:04.246001 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:35:04.246001 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:35:04.247595 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:35:04.247595 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:35:04.247595 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:35:04.247595 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:35:04.247595 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:04.247595 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:04.247595 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:04.247595 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:04.247595 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:04.247595 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:04.247595 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:04.247595 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:04.247595 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:04.247595 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:04.247595 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:04.247595 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:04.248508 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:04.248508 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:04.248508 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:04.248508 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:04.248508 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:35:04.248508 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:35:04.248508 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:35:04.248508 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:35:04.248508 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:35:04.248508 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:35:04.248508 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:35:04.248508 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:35:04.251751 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 387/782] 10.222.0.55 () {56 vars in 1096 bytes} [Wed Aug 24 08:35:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:04.253848 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ba2e70-b4a8-4c1f-a238-290a058d6fad None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:04.254341 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ba2e70-b4a8-4c1f-a238-290a058d6fad None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:04.254827 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ba2e70-b4a8-4c1f-a238-290a058d6fad None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:04.259124 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 388/783] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:35:04 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:35:04.260572 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ba2e70-b4a8-4c1f-a238-290a058d6fad None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:04.302593 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f4d22f-329b-4a42-b108-abd4a96e28cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:04.303751 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f4d22f-329b-4a42-b108-abd4a96e28cf None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:04.304221 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f4d22f-329b-4a42-b108-abd4a96e28cf None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:04.304783 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f4d22f-329b-4a42-b108-abd4a96e28cf None admin] PATH_INFO: `/v3/services` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:04.305883 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f4d22f-329b-4a42-b108-abd4a96e28cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:04.309364 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f4d22f-329b-4a42-b108-abd4a96e28cf None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:04.328575 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 396/784] 10.222.0.55 () {56 vars in 1087 bytes} [Wed Aug 24 08:35:04 2022] GET /identity/v3/services => generated 772 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:04.328575 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d8a878-b0d7-425c-9ebb-4798e2eae532 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:04.328575 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d8a878-b0d7-425c-9ebb-4798e2eae532 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:04.328575 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d8a878-b0d7-425c-9ebb-4798e2eae532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:04.374636 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-e3d8a878-b0d7-425c-9ebb-4798e2eae532 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:35:04.378279 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-e3d8a878-b0d7-425c-9ebb-4798e2eae532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:35:04.474129 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:04.477382 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ba2e70-b4a8-4c1f-a238-290a058d6fad None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:04.494375 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 397/785] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:35:04 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:35:04.542727 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322467c4-bf44-4fe7-9ada-b0fffd29f663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:04.544701 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322467c4-bf44-4fe7-9ada-b0fffd29f663 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:04.545430 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322467c4-bf44-4fe7-9ada-b0fffd29f663 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:04.546164 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322467c4-bf44-4fe7-9ada-b0fffd29f663 None admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:04.547699 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322467c4-bf44-4fe7-9ada-b0fffd29f663 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:04.548485 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322467c4-bf44-4fe7-9ada-b0fffd29f663 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:04.551409 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322467c4-bf44-4fe7-9ada-b0fffd29f663 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:04.567374 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 389/786] 10.222.0.55 () {56 vars in 1099 bytes} [Wed Aug 24 08:35:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:04.577163 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d8a878-b0d7-425c-9ebb-4798e2eae532 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:04.715513 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d8a878-b0d7-425c-9ebb-4798e2eae532 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:04.769988 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2239df-e12d-4189-a2d6-910e68641e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:04.770535 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2239df-e12d-4189-a2d6-910e68641e7b None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:04.770817 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2239df-e12d-4189-a2d6-910e68641e7b None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:04.771090 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2239df-e12d-4189-a2d6-910e68641e7b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:04.790581 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2239df-e12d-4189-a2d6-910e68641e7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:04.793735 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2239df-e12d-4189-a2d6-910e68641e7b None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:04.805210 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322467c4-bf44-4fe7-9ada-b0fffd29f663 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:04.822216 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-cf2239df-e12d-4189-a2d6-910e68641e7b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:04.822216 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:04.822216 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:04.822216 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:04.822216 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:04.822216 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:04.822216 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:04.822216 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:04.822216 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:04.822216 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:04.822216 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:04.822216 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:04.822216 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:35:04.822216 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:35:04.822216 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:35:04.822216 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:35:04.823240 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:04.823240 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:04.823240 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:04.823240 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:04.823240 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:04.823240 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:04.823240 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:04.823240 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:04.823240 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:04.823240 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:04.823240 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:04.823240 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:04.823240 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:04.823240 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:04.823240 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:04.823240 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:04.823240 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:35:04.824151 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:35:04.824151 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:04.824151 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:35:04.827571 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 398/787] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:35:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:04.833453 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2239df-e12d-4189-a2d6-910e68641e7b None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:04.890418 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:04.914658 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:05.021850 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322467c4-bf44-4fe7-9ada-b0fffd29f663 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:05.058416 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2239df-e12d-4189-a2d6-910e68641e7b None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:05.067639 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7870b5-66f7-4963-853a-ca7319e2d2e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:05.067639 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7870b5-66f7-4963-853a-ca7319e2d2e2 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:05.067639 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7870b5-66f7-4963-853a-ca7319e2d2e2 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:05.067639 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7870b5-66f7-4963-853a-ca7319e2d2e2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:05.079174 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7870b5-66f7-4963-853a-ca7319e2d2e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:35:05.086163 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-7b7870b5-66f7-4963-853a-ca7319e2d2e2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:35:05.086163 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:05.086163 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:05.086163 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:05.086163 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:05.086163 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:05.086163 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:05.086163 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:05.086163 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:05.086163 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:05.086163 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:05.086163 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:05.086163 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:35:05.086163 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:35:05.086163 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:35:05.086163 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:35:05.087540 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:35:05.087540 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:35:05.087540 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:35:05.087540 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:35:05.087540 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:05.087540 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:05.087540 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:05.087540 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:05.087540 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:05.087540 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:05.087540 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:05.087540 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:05.087540 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:05.087540 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:05.087540 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:05.087540 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:05.089871 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:05.089871 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:05.089871 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:05.089871 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:05.089871 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:35:05.089871 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:35:05.089871 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:35:05.089871 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:35:05.089871 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:35:05.089871 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:35:05.089871 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:35:05.089871 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:35:05.089871 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 390/788] 10.222.0.55 () {56 vars in 1102 bytes} [Wed Aug 24 08:35:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:35:05.089871 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34c1243-db94-4840-a061-a95168b794e4 None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:05.092594 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34c1243-db94-4840-a061-a95168b794e4 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:05.092594 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34c1243-db94-4840-a061-a95168b794e4 None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:05.092594 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 391/789] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:35:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:35:05.098037 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426fee18-7c0e-4498-b803-f3db03bd823d None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:05.098672 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426fee18-7c0e-4498-b803-f3db03bd823d None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:05.099390 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426fee18-7c0e-4498-b803-f3db03bd823d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:05.125986 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558b1ff3-76a3-4660-bfff-333c2a26ec9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:05.126767 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558b1ff3-76a3-4660-bfff-333c2a26ec9a None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:05.127122 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558b1ff3-76a3-4660-bfff-333c2a26ec9a None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:05.127463 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558b1ff3-76a3-4660-bfff-333c2a26ec9a None admin] PATH_INFO: `/v3/domains` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:05.127792 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-426fee18-7c0e-4498-b803-f3db03bd823d None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:35:05.128263 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558b1ff3-76a3-4660-bfff-333c2a26ec9a None admin] RBAC: Adding query filter params (name=Default) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:05.128668 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558b1ff3-76a3-4660-bfff-333c2a26ec9a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:05.130068 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-426fee18-7c0e-4498-b803-f3db03bd823d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:35:05.130444 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558b1ff3-76a3-4660-bfff-333c2a26ec9a None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:05.147729 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 399/790] 10.222.0.55 () {56 vars in 1109 bytes} [Wed Aug 24 08:35:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:05.148775 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558b1ff3-76a3-4660-bfff-333c2a26ec9a None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:05.229246 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 392/791] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:35:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:35:05.230397 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426fee18-7c0e-4498-b803-f3db03bd823d None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:05.317002 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:05.404646 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:05.409198 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558b1ff3-76a3-4660-bfff-333c2a26ec9a None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:05.487668 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aa9323-14d3-47a8-87a3-c575962eee31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:05.488368 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aa9323-14d3-47a8-87a3-c575962eee31 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:05.488731 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aa9323-14d3-47a8-87a3-c575962eee31 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:05.489069 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aa9323-14d3-47a8-87a3-c575962eee31 None admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:05.489687 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0aa9323-14d3-47a8-87a3-c575962eee31 None admin] RBAC: Adding query filter params (name=service) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:05.489987 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0aa9323-14d3-47a8-87a3-c575962eee31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:05.491429 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0aa9323-14d3-47a8-87a3-c575962eee31 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:05.502202 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 400/792] 10.222.0.55 () {56 vars in 1103 bytes} [Wed Aug 24 08:35:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:05.517296 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2897e2ab-57b7-43f5-9539-443bd228e99b None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:05.517296 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2897e2ab-57b7-43f5-9539-443bd228e99b None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:05.517296 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2897e2ab-57b7-43f5-9539-443bd228e99b None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:05.517296 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 401/793] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:35:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:35:05.517296 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2897e2ab-57b7-43f5-9539-443bd228e99b None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:05.527140 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426fee18-7c0e-4498-b803-f3db03bd823d None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:05.568691 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52606a0e-eca6-4306-b270-64bd9c277f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:05.568691 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52606a0e-eca6-4306-b270-64bd9c277f6a None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:05.568691 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52606a0e-eca6-4306-b270-64bd9c277f6a None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:05.568691 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52606a0e-eca6-4306-b270-64bd9c277f6a None admin] PATH_INFO: `/v3/users/swift` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:05.578733 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52606a0e-eca6-4306-b270-64bd9c277f6a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:05.580201 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52606a0e-eca6-4306-b270-64bd9c277f6a None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:05.594589 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-52606a0e-eca6-4306-b270-64bd9c277f6a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 08:35:05.594589 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:05.594589 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:05.594589 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:05.594589 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:05.594589 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:05.594589 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:05.594589 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:05.594589 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:05.594589 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:05.594589 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:05.594589 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:05.594589 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:35:05.594589 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:35:05.594589 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:35:05.594589 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:35:05.596745 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:05.596745 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:05.596745 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:35:05.596745 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:35:05.596745 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:35:05.596745 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:35:05.596745 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:05.596745 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:05.596745 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:05.596745 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:05.596745 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:05.596745 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:05.596745 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:05.596745 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:05.596745 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:05.596745 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:05.596745 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:05.596745 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:05.598100 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:05.598100 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:05.598100 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:35:05.598100 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:35:05.598100 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:35:05.598100 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:35:05.598100 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:35:05.598100 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:35:05.598100 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 08:35:05.598100 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:35:05.598100 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 393/794] 10.222.0.55 () {56 vars in 1096 bytes} [Wed Aug 24 08:35:05 2022] GET /identity/v3/users/swift => generated 83 bytes in 366 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:05.598100 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5bcf1b-fa54-460d-9675-583f329cf072 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:05.598100 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5bcf1b-fa54-460d-9675-583f329cf072 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:05.599034 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5bcf1b-fa54-460d-9675-583f329cf072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:05.618002 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-7c5bcf1b-fa54-460d-9675-583f329cf072 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:35:05.619194 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-7c5bcf1b-fa54-460d-9675-583f329cf072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:35:05.676510 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2897e2ab-57b7-43f5-9539-443bd228e99b None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:05.695178 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 394/795] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:35:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:35:05.696497 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5bcf1b-fa54-460d-9675-583f329cf072 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:05.712001 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:05.734135 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b68009-ee36-4684-9e55-d52fb553729e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:05.736067 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b68009-ee36-4684-9e55-d52fb553729e None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:05.736569 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b68009-ee36-4684-9e55-d52fb553729e None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:05.737005 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b68009-ee36-4684-9e55-d52fb553729e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:05.752823 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b68009-ee36-4684-9e55-d52fb553729e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:05.755095 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b68009-ee36-4684-9e55-d52fb553729e None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:05.776167 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-24b68009-ee36-4684-9e55-d52fb553729e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:05.776167 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:05.776167 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:05.776167 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:05.776167 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:05.776167 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:05.776167 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:05.776167 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:05.776167 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:05.776167 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:05.776167 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:05.776167 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:05.776167 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:35:05.776167 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:35:05.776167 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:35:05.776167 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:35:05.777420 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:05.777420 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:05.777420 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:05.777420 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:05.777420 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:05.777420 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:05.777420 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:05.777420 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:05.777420 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:05.777420 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:05.777420 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:05.777420 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:05.777420 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:05.777420 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:05.777420 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:05.777420 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:05.777420 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:35:05.779099 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:35:05.779099 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:05.779099 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:35:05.780937 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 402/796] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:35:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:05.782027 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b68009-ee36-4684-9e55-d52fb553729e None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:05.809333 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:05.831000 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5bcf1b-fa54-460d-9675-583f329cf072 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:05.877775 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab5e9dc-4e19-4872-b79a-a8cd31ebfb3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:05.878436 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab5e9dc-4e19-4872-b79a-a8cd31ebfb3e None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:05.878720 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab5e9dc-4e19-4872-b79a-a8cd31ebfb3e None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:05.879001 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab5e9dc-4e19-4872-b79a-a8cd31ebfb3e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:05.892912 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab5e9dc-4e19-4872-b79a-a8cd31ebfb3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:35:05.897854 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-eab5e9dc-4e19-4872-b79a-a8cd31ebfb3e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:35:05.897854 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:05.897854 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:05.897854 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:05.897854 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:05.897854 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:05.897854 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:05.897854 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:05.897854 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:05.897854 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:05.897854 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:05.897854 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:05.897854 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:35:05.897854 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:35:05.897854 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:35:05.897854 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:35:05.899550 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:35:05.899550 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:35:05.899550 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:35:05.899550 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:35:05.899550 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:05.899550 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:05.899550 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:05.899550 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:05.899550 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:05.899550 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:05.899550 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:05.899550 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:05.899550 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:05.899550 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:05.899550 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:05.899550 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:05.901863 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:05.901863 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:05.901863 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:05.901863 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:05.901863 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:35:05.901863 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:35:05.901863 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:35:05.901863 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:35:05.901863 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:35:05.901863 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:35:05.901863 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:35:05.901863 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:35:05.904233 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 395/797] 10.222.0.55 () {56 vars in 1096 bytes} [Wed Aug 24 08:35:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:05.906751 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab5e9dc-4e19-4872-b79a-a8cd31ebfb3e None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:06.005758 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b68009-ee36-4684-9e55-d52fb553729e None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:06.062523 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701c7732-788d-4d12-9217-f2f4ad5d5d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:06.063154 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701c7732-788d-4d12-9217-f2f4ad5d5d3c None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:06.063436 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701c7732-788d-4d12-9217-f2f4ad5d5d3c None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:06.063701 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701c7732-788d-4d12-9217-f2f4ad5d5d3c None admin] PATH_INFO: `/v3/users` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:06.064462 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701c7732-788d-4d12-9217-f2f4ad5d5d3c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:06.064806 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701c7732-788d-4d12-9217-f2f4ad5d5d3c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:06.066299 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701c7732-788d-4d12-9217-f2f4ad5d5d3c None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:06.091351 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 403/798] 10.222.0.55 () {56 vars in 1135 bytes} [Wed Aug 24 08:35:05 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:06.091922 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701c7732-788d-4d12-9217-f2f4ad5d5d3c None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:06.120043 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:06.158020 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab5e9dc-4e19-4872-b79a-a8cd31ebfb3e None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:06.203881 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:06.214931 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4a61d6-1f9e-4682-abeb-28552079c076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:06.215627 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4a61d6-1f9e-4682-abeb-28552079c076 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:06.216320 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4a61d6-1f9e-4682-abeb-28552079c076 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:06.216320 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4a61d6-1f9e-4682-abeb-28552079c076 None admin] PATH_INFO: `/v3/domains` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:06.216991 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4a61d6-1f9e-4682-abeb-28552079c076 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:06.217253 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4a61d6-1f9e-4682-abeb-28552079c076 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:06.218665 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4a61d6-1f9e-4682-abeb-28552079c076 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:06.239950 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 396/799] 10.222.0.55 () {56 vars in 1109 bytes} [Wed Aug 24 08:35:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:06.239950 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4a61d6-1f9e-4682-abeb-28552079c076 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:06.251450 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701c7732-788d-4d12-9217-f2f4ad5d5d3c None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:06.296683 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ad821f-3117-4736-bf67-693cbd155686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:06.297307 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ad821f-3117-4736-bf67-693cbd155686 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:06.297576 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ad821f-3117-4736-bf67-693cbd155686 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:06.297848 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ad821f-3117-4736-bf67-693cbd155686 None admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:06.298525 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ad821f-3117-4736-bf67-693cbd155686 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:06.298822 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ad821f-3117-4736-bf67-693cbd155686 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:06.300217 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ad821f-3117-4736-bf67-693cbd155686 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:06.310911 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 404/800] 10.222.0.55 () {56 vars in 1099 bytes} [Wed Aug 24 08:35:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:06.311550 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ad821f-3117-4736-bf67-693cbd155686 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:06.396369 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4a61d6-1f9e-4682-abeb-28552079c076 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:06.436083 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:06.443069 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95881c87-ac49-4675-9f78-ce8e3aa58d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:06.444498 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95881c87-ac49-4675-9f78-ce8e3aa58d1b None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:06.460879 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95881c87-ac49-4675-9f78-ce8e3aa58d1b None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:06.460879 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95881c87-ac49-4675-9f78-ce8e3aa58d1b None admin] PATH_INFO: `/v3/services/network` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:06.460879 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95881c87-ac49-4675-9f78-ce8e3aa58d1b None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:06.460879 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95881c87-ac49-4675-9f78-ce8e3aa58d1b None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:06.461237 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ad821f-3117-4736-bf67-693cbd155686 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:06.465403 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-95881c87-ac49-4675-9f78-ce8e3aa58d1b None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 08:35:06.465403 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:06.465403 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 08:35:06.465403 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 08:35:06.465403 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 08:35:06.465403 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 08:35:06.465403 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 08:35:06.465403 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:35:06.465403 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 08:35:06.465403 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:35:06.465403 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:35:06.465403 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:35:06.465403 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:06.465403 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:06.465403 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:06.465403 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:06.465403 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:06.466485 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:06.466485 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:06.466485 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:06.466485 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:06.466485 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:06.466485 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:06.466485 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 08:35:06.466485 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 08:35:06.466485 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 08:35:06.466485 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 08:35:06.466485 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:06.466485 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:06.466485 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:06.466485 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:06.466485 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:06.466485 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:06.467404 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:06.467404 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:06.467404 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:06.467404 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:06.467404 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:06.467404 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:06.467404 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:06.467404 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:06.467404 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:06.467404 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:06.467404 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 08:35:06.467404 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:35:06.467404 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 08:35:06.467404 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:35:06.468196 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 397/801] 10.222.0.55 () {56 vars in 1111 bytes} [Wed Aug 24 08:35:06 2022] GET /identity/v3/services/network => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:06.469222 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95881c87-ac49-4675-9f78-ce8e3aa58d1b None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:06.514967 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea3cd78-ed50-46dc-962b-7b896c8cdc03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:06.515757 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea3cd78-ed50-46dc-962b-7b896c8cdc03 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:06.516043 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea3cd78-ed50-46dc-962b-7b896c8cdc03 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:06.516317 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea3cd78-ed50-46dc-962b-7b896c8cdc03 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:06.519138 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:06.531739 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea3cd78-ed50-46dc-962b-7b896c8cdc03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:06.533276 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea3cd78-ed50-46dc-962b-7b896c8cdc03 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:06.549821 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-2ea3cd78-ed50-46dc-962b-7b896c8cdc03 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:06.549821 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:06.549821 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:06.549821 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:06.549821 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:06.549821 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:06.549821 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:06.549821 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:06.549821 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:06.549821 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:06.549821 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:06.549821 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:06.549821 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:35:06.549821 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:35:06.549821 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:35:06.549821 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:35:06.551531 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:06.551531 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:06.551531 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:06.551531 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:06.551531 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:06.551531 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:06.551531 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:06.551531 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:06.551531 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:06.551531 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:06.551531 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:06.551531 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:06.551531 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:06.551531 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:06.551531 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:06.551531 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:06.551531 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:35:06.552742 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:35:06.552742 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:06.552742 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:35:06.552742 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 405/802] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:35:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:06.553124 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea3cd78-ed50-46dc-962b-7b896c8cdc03 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:06.613026 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95881c87-ac49-4675-9f78-ce8e3aa58d1b None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:06.653753 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7a34e4-1d57-4396-9dc1-0540f18c67c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:06.654326 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7a34e4-1d57-4396-9dc1-0540f18c67c5 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:06.654602 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7a34e4-1d57-4396-9dc1-0540f18c67c5 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:06.654874 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7a34e4-1d57-4396-9dc1-0540f18c67c5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:06.664744 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7a34e4-1d57-4396-9dc1-0540f18c67c5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:06.666655 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7a34e4-1d57-4396-9dc1-0540f18c67c5 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:06.683964 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-3e7a34e4-1d57-4396-9dc1-0540f18c67c5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 08:35:06.683964 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:06.683964 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:06.683964 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:06.683964 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:06.683964 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:06.683964 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:06.683964 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:06.683964 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:06.683964 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:06.683964 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:06.683964 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:06.683964 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:35:06.683964 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:35:06.683964 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:35:06.683964 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:35:06.685918 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:06.685918 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:06.685918 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:35:06.685918 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:35:06.685918 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:35:06.685918 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:35:06.685918 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:06.685918 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:06.685918 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:06.685918 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:06.685918 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:06.685918 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:06.685918 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:06.685918 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:06.685918 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:06.685918 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:06.685918 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:06.685918 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:06.687427 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:06.687427 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:06.687427 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:35:06.687427 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:35:06.687427 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:35:06.687427 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:35:06.687427 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:35:06.687427 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:35:06.687427 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 08:35:06.687427 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:35:06.687427 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 398/803] 10.222.0.55 () {56 vars in 1117 bytes} [Wed Aug 24 08:35:06 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:06.687427 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7a34e4-1d57-4396-9dc1-0540f18c67c5 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:06.703843 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea3cd78-ed50-46dc-962b-7b896c8cdc03 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:06.753793 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:06.755163 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db0f808-020a-4c64-a5c4-e230ec53a6ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:06.756511 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db0f808-020a-4c64-a5c4-e230ec53a6ab None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:06.757007 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db0f808-020a-4c64-a5c4-e230ec53a6ab None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:06.757487 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db0f808-020a-4c64-a5c4-e230ec53a6ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:06.781790 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db0f808-020a-4c64-a5c4-e230ec53a6ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:06.784364 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db0f808-020a-4c64-a5c4-e230ec53a6ab None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:06.801531 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-7db0f808-020a-4c64-a5c4-e230ec53a6ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:06.801531 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:06.801531 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:06.801531 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:06.801531 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:06.801531 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:06.801531 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:06.801531 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:06.801531 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:06.801531 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:06.801531 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:06.801531 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:06.801531 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:35:06.801531 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:35:06.801531 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:35:06.801531 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:35:06.802631 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:06.802631 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:06.802631 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:06.802631 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:06.802631 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:06.802631 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:06.802631 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:06.802631 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:06.802631 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:06.802631 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:06.802631 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:06.802631 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:06.802631 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:06.802631 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:06.802631 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:06.802631 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:06.802631 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:35:06.803932 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:35:06.803932 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:06.803932 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:35:06.803932 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 406/804] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:35:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:06.803932 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db0f808-020a-4c64-a5c4-e230ec53a6ab None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:06.809194 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7a34e4-1d57-4396-9dc1-0540f18c67c5 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:06.848552 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb0a0a0-2daf-43e6-a4cc-f577fdb449f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:06.849232 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb0a0a0-2daf-43e6-a4cc-f577fdb449f7 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:06.849454 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb0a0a0-2daf-43e6-a4cc-f577fdb449f7 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:06.849725 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb0a0a0-2daf-43e6-a4cc-f577fdb449f7 None admin] PATH_INFO: `/v3/services` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:06.850360 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb0a0a0-2daf-43e6-a4cc-f577fdb449f7 None admin] RBAC: Adding query filter params (name=network) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:06.850661 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb0a0a0-2daf-43e6-a4cc-f577fdb449f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:06.852028 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb0a0a0-2daf-43e6-a4cc-f577fdb449f7 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:06.852646 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:06.857457 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 399/805] 10.222.0.55 () {56 vars in 1112 bytes} [Wed Aug 24 08:35:06 2022] GET /identity/v3/services?name=network => generated 124 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:06.858301 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb0a0a0-2daf-43e6-a4cc-f577fdb449f7 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:06.927423 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db0f808-020a-4c64-a5c4-e230ec53a6ab None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:06.962790 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5084a770-d5f4-4a0d-9ae9-494a0b483ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:06.963398 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5084a770-d5f4-4a0d-9ae9-494a0b483ed5 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:06.963676 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5084a770-d5f4-4a0d-9ae9-494a0b483ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:06.963948 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5084a770-d5f4-4a0d-9ae9-494a0b483ed5 None admin] PATH_INFO: `/v3/domains` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:06.964632 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5084a770-d5f4-4a0d-9ae9-494a0b483ed5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:06.964935 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5084a770-d5f4-4a0d-9ae9-494a0b483ed5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:06.966814 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5084a770-d5f4-4a0d-9ae9-494a0b483ed5 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:06.979300 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 407/806] 10.222.0.55 () {56 vars in 1109 bytes} [Wed Aug 24 08:35:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:06.979752 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb0a0a0-2daf-43e6-a4cc-f577fdb449f7 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:06.980557 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33346dbd-91c4-4e7c-8449-7fa0918dc60a None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:06.980842 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33346dbd-91c4-4e7c-8449-7fa0918dc60a None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:06.981112 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33346dbd-91c4-4e7c-8449-7fa0918dc60a None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:06.983007 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 408/807] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:35:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:35:06.983720 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33346dbd-91c4-4e7c-8449-7fa0918dc60a None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:07.002754 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:07.018367 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5037618f-8758-47c1-ba41-e6bf3804a095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:07.018997 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5037618f-8758-47c1-ba41-e6bf3804a095 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:07.019268 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5037618f-8758-47c1-ba41-e6bf3804a095 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:07.019533 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5037618f-8758-47c1-ba41-e6bf3804a095 None admin] PATH_INFO: `/v3/users` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:07.020413 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5037618f-8758-47c1-ba41-e6bf3804a095 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:07.020764 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5037618f-8758-47c1-ba41-e6bf3804a095 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:07.022219 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5037618f-8758-47c1-ba41-e6bf3804a095 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:07.048336 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 400/808] 10.222.0.55 () {56 vars in 1149 bytes} [Wed Aug 24 08:35:06 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:07.048336 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5037618f-8758-47c1-ba41-e6bf3804a095 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:07.093862 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:07.099097 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33346dbd-91c4-4e7c-8449-7fa0918dc60a None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:07.134701 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1395428b-41b1-4e2a-8485-9686f00b18ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:07.135308 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1395428b-41b1-4e2a-8485-9686f00b18ca None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:07.135583 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1395428b-41b1-4e2a-8485-9686f00b18ca None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:07.135853 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1395428b-41b1-4e2a-8485-9686f00b18ca None admin] PATH_INFO: `/v3/domains` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:07.136488 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1395428b-41b1-4e2a-8485-9686f00b18ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:07.136824 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1395428b-41b1-4e2a-8485-9686f00b18ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:07.138693 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1395428b-41b1-4e2a-8485-9686f00b18ca None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:07.150968 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 409/809] 10.222.0.55 () {56 vars in 1109 bytes} [Wed Aug 24 08:35:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:07.151806 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1395428b-41b1-4e2a-8485-9686f00b18ca None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:07.160048 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5037618f-8758-47c1-ba41-e6bf3804a095 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:07.194325 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec774397-7ae8-4fbb-84d1-07c5d90f354d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:07.194884 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec774397-7ae8-4fbb-84d1-07c5d90f354d None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:07.195157 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec774397-7ae8-4fbb-84d1-07c5d90f354d None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:07.195423 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec774397-7ae8-4fbb-84d1-07c5d90f354d None admin] PATH_INFO: `/v3/services` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:07.196064 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec774397-7ae8-4fbb-84d1-07c5d90f354d None admin] RBAC: Adding query filter params (type=network) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:07.196357 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec774397-7ae8-4fbb-84d1-07c5d90f354d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:07.197760 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec774397-7ae8-4fbb-84d1-07c5d90f354d None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:07.203660 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 401/810] 10.222.0.55 () {56 vars in 1112 bytes} [Wed Aug 24 08:35:07 2022] GET /identity/v3/services?type=network => generated 350 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:07.204908 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bcc886-c7f3-4f6b-9711-11aa20132c09 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:07.205186 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bcc886-c7f3-4f6b-9711-11aa20132c09 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:07.205455 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bcc886-c7f3-4f6b-9711-11aa20132c09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:07.223514 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-50bcc886-c7f3-4f6b-9711-11aa20132c09 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:35:07.224735 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-50bcc886-c7f3-4f6b-9711-11aa20132c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:35:07.261342 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:07.268314 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1395428b-41b1-4e2a-8485-9686f00b18ca None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:07.295641 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 402/811] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:35:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:35:07.296270 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bcc886-c7f3-4f6b-9711-11aa20132c09 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:07.322064 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55ba846-cd41-4bad-b614-85269f3b31e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:07.322714 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55ba846-cd41-4bad-b614-85269f3b31e5 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:07.323142 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55ba846-cd41-4bad-b614-85269f3b31e5 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:07.323418 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55ba846-cd41-4bad-b614-85269f3b31e5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:07.333369 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55ba846-cd41-4bad-b614-85269f3b31e5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:07.334809 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55ba846-cd41-4bad-b614-85269f3b31e5 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:07.351518 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-c55ba846-cd41-4bad-b614-85269f3b31e5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:35:07.351518 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:07.351518 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:07.351518 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:07.351518 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:07.351518 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:07.351518 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:07.351518 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:07.351518 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:07.351518 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:07.351518 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:07.351518 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:07.351518 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:35:07.351518 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:35:07.351518 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:35:07.352512 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:35:07.352512 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:07.352512 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:07.352512 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:07.352512 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:07.352512 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:07.352512 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:07.352512 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:07.352512 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:07.352512 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:07.352512 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:07.352512 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:07.352512 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:07.352512 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:07.352512 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:07.352512 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:07.352512 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:07.358889 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:35:07.358889 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:35:07.358889 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:35:07.358889 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:35:07.358889 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:35:07.358889 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:35:07.358889 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:35:07.358889 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:35:07.358889 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 410/812] 10.222.0.55 () {56 vars in 1111 bytes} [Wed Aug 24 08:35:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:07.358889 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55ba846-cd41-4bad-b614-85269f3b31e5 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:07.385927 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:07.406579 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bcc886-c7f3-4f6b-9711-11aa20132c09 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:07.446479 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb86847-f943-4dd8-b939-4f6b98a71d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:07.447030 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb86847-f943-4dd8-b939-4f6b98a71d8a None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:07.447303 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb86847-f943-4dd8-b939-4f6b98a71d8a None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:07.447567 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb86847-f943-4dd8-b939-4f6b98a71d8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:07.459874 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb86847-f943-4dd8-b939-4f6b98a71d8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:07.461481 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb86847-f943-4dd8-b939-4f6b98a71d8a None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:07.476529 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-dbb86847-f943-4dd8-b939-4f6b98a71d8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:07.476529 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:07.476529 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:07.476529 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:07.476529 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:07.476529 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:07.476529 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:07.476529 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:07.476529 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:07.476529 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:07.476529 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:07.476529 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:07.476529 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:35:07.476529 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:35:07.476529 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:35:07.476529 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:35:07.477602 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:07.477602 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:07.477602 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:07.477602 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:07.477602 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:07.477602 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:07.477602 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:07.477602 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:07.477602 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:07.477602 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:07.477602 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:07.477602 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:07.477602 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:07.477602 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:07.477602 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:07.477602 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:07.477602 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:35:07.478562 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:35:07.478562 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:07.478562 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:35:07.478562 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 403/813] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:35:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:07.478562 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb86847-f943-4dd8-b939-4f6b98a71d8a None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:07.497498 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55ba846-cd41-4bad-b614-85269f3b31e5 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:07.510748 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:07.545380 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76576d20-35d1-48d3-a8d4-e3e0b556e6b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:07.546018 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76576d20-35d1-48d3-a8d4-e3e0b556e6b2 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:07.546301 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76576d20-35d1-48d3-a8d4-e3e0b556e6b2 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:07.546572 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76576d20-35d1-48d3-a8d4-e3e0b556e6b2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:07.555494 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76576d20-35d1-48d3-a8d4-e3e0b556e6b2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:07.557758 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76576d20-35d1-48d3-a8d4-e3e0b556e6b2 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:07.574322 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-76576d20-35d1-48d3-a8d4-e3e0b556e6b2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 08:35:07.574322 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:07.574322 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:07.574322 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:07.574322 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:07.574322 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:07.574322 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:07.574322 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:07.574322 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:07.574322 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:07.574322 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:07.574322 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:07.574322 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:35:07.574322 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:35:07.574322 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:35:07.574322 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:35:07.576646 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:07.576646 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:07.576646 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:35:07.576646 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:35:07.576646 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:35:07.576646 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:35:07.576646 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:07.576646 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:07.576646 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:07.576646 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:07.576646 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:07.576646 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:07.576646 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:07.576646 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:07.576646 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:07.576646 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:07.576646 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:07.576646 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:07.578741 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:07.578741 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:07.578741 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:35:07.578741 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:35:07.578741 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:35:07.578741 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:35:07.578741 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:35:07.578741 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:35:07.578741 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 08:35:07.578741 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:35:07.578741 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 411/814] 10.222.0.55 () {56 vars in 1093 bytes} [Wed Aug 24 08:35:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:07.578741 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76576d20-35d1-48d3-a8d4-e3e0b556e6b2 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:07.611435 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb86847-f943-4dd8-b939-4f6b98a71d8a None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:07.649705 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d25adc3-24f6-4ecf-b67c-572cd1dd1f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:07.650317 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d25adc3-24f6-4ecf-b67c-572cd1dd1f91 None admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:07.650735 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d25adc3-24f6-4ecf-b67c-572cd1dd1f91 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:07.651193 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d25adc3-24f6-4ecf-b67c-572cd1dd1f91 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:07.652283 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d25adc3-24f6-4ecf-b67c-572cd1dd1f91 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:07.654683 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d25adc3-24f6-4ecf-b67c-572cd1dd1f91 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:07.674174 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:07.697200 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76576d20-35d1-48d3-a8d4-e3e0b556e6b2 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:07.734649 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d78f5f-b3d8-42aa-8095-5775ed294f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:07.735265 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d78f5f-b3d8-42aa-8095-5775ed294f0e None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:07.735547 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d78f5f-b3d8-42aa-8095-5775ed294f0e None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:07.735825 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d78f5f-b3d8-42aa-8095-5775ed294f0e None admin] PATH_INFO: `/v3/projects` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:07.736528 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d78f5f-b3d8-42aa-8095-5775ed294f0e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:07.736863 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d78f5f-b3d8-42aa-8095-5775ed294f0e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:07.738316 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d78f5f-b3d8-42aa-8095-5775ed294f0e None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:07.752771 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 412/815] 10.222.0.55 () {56 vars in 1148 bytes} [Wed Aug 24 08:35:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:07.753743 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d78f5f-b3d8-42aa-8095-5775ed294f0e None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:07.764809 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 404/816] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 08:35:07 2022] POST /identity/v3/endpoints => generated 337 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:35:07.769544 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d25adc3-24f6-4ecf-b67c-572cd1dd1f91 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:07.890976 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:07.934132 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d25adc3-24f6-4ecf-b67c-572cd1dd1f91 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:07.950980 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d78f5f-b3d8-42aa-8095-5775ed294f0e None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:07.986977 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57523966-da15-4569-bb0b-c7458c8943fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:07.988019 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57523966-da15-4569-bb0b-c7458c8943fe None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:07.988483 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57523966-da15-4569-bb0b-c7458c8943fe None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:07.989797 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57523966-da15-4569-bb0b-c7458c8943fe None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:07.991247 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57523966-da15-4569-bb0b-c7458c8943fe None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:07.994431 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57523966-da15-4569-bb0b-c7458c8943fe None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:08.006583 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-57523966-da15-4569-bb0b-c7458c8943fe None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 08:35:08.006583 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:08.006583 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 08:35:08.006583 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 08:35:08.006583 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 08:35:08.006583 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 08:35:08.006583 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 08:35:08.006583 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:35:08.006583 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 08:35:08.006583 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:35:08.006583 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:35:08.006583 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:35:08.006583 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:08.006583 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:08.006583 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:08.006583 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:08.006583 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:08.008154 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:08.008154 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:08.008154 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:08.008154 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:08.008154 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:08.008154 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:08.008154 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 08:35:08.008154 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 08:35:08.008154 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 08:35:08.008154 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 08:35:08.008154 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:08.008154 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:08.008154 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:08.008154 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:08.008154 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:08.008154 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:08.010566 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:08.010566 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:08.010566 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:08.010566 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:08.010566 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:08.010566 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:08.010566 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:08.010566 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:08.010566 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:08.010566 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:08.010566 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 08:35:08.010566 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:35:08.010566 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 08:35:08.010566 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:35:08.010566 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 405/817] 10.222.0.55 () {56 vars in 1129 bytes} [Wed Aug 24 08:35:07 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:08.012114 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57523966-da15-4569-bb0b-c7458c8943fe None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:08.017427 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df95ec9-8ab0-4d2c-84c0-52742e29828f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:08.018380 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df95ec9-8ab0-4d2c-84c0-52742e29828f None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:08.018855 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df95ec9-8ab0-4d2c-84c0-52742e29828f None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:08.019314 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df95ec9-8ab0-4d2c-84c0-52742e29828f None admin] PATH_INFO: `/v3/domains` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:08.020429 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df95ec9-8ab0-4d2c-84c0-52742e29828f None admin] RBAC: Adding query filter params (name=Default) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:08.020959 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df95ec9-8ab0-4d2c-84c0-52742e29828f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:08.023371 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df95ec9-8ab0-4d2c-84c0-52742e29828f None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:08.036464 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:08.045924 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 413/818] 10.222.0.55 () {56 vars in 1109 bytes} [Wed Aug 24 08:35:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:08.052845 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df95ec9-8ab0-4d2c-84c0-52742e29828f None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:08.131930 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57523966-da15-4569-bb0b-c7458c8943fe None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:08.175100 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ad4d55-200f-4e6b-b308-8d10fb2de265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:08.175903 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ad4d55-200f-4e6b-b308-8d10fb2de265 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:08.176242 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ad4d55-200f-4e6b-b308-8d10fb2de265 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:08.176658 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ad4d55-200f-4e6b-b308-8d10fb2de265 None admin] PATH_INFO: `/v3/users` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:08.177522 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ad4d55-200f-4e6b-b308-8d10fb2de265 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:08.177956 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ad4d55-200f-4e6b-b308-8d10fb2de265 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:08.179667 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ad4d55-200f-4e6b-b308-8d10fb2de265 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:08.183411 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df95ec9-8ab0-4d2c-84c0-52742e29828f None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:08.191364 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:08.204229 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 406/819] 10.222.0.55 () {56 vars in 1133 bytes} [Wed Aug 24 08:35:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:08.207027 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ad4d55-200f-4e6b-b308-8d10fb2de265 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:08.231496 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95deb5ce-1e5d-466f-b734-bda25fadc87f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:08.232939 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95deb5ce-1e5d-466f-b734-bda25fadc87f None admin] REQUEST_METHOD: `PUT` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:08.233305 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95deb5ce-1e5d-466f-b734-bda25fadc87f None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:08.233662 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95deb5ce-1e5d-466f-b734-bda25fadc87f None admin] PATH_INFO: `/v3/projects/420b18725f2342159646fb86438318fd/users/a0ebb5ccabef45378f0c1da69721e051/roles/b870cd1acaa744e1ae1d58b9ac283821` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:08.235826 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95deb5ce-1e5d-466f-b734-bda25fadc87f None admin] RBAC: Authorizing `identity:create_grant(project_id=420b18725f2342159646fb86438318fd, user_id=a0ebb5ccabef45378f0c1da69721e051, role_id=b870cd1acaa744e1ae1d58b9ac283821)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:08.237629 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95deb5ce-1e5d-466f-b734-bda25fadc87f None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:08.313298 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 414/820] 10.222.0.55 () {58 vars in 1439 bytes} [Wed Aug 24 08:35:08 2022] PUT /identity/v3/projects/420b18725f2342159646fb86438318fd/users/a0ebb5ccabef45378f0c1da69721e051/roles/b870cd1acaa744e1ae1d58b9ac283821 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:35:08.319782 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ad4d55-200f-4e6b-b308-8d10fb2de265 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:08.328634 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95deb5ce-1e5d-466f-b734-bda25fadc87f None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:08.378135 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e43b83a-4533-4493-94cd-ae44caf91c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:08.379110 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e43b83a-4533-4493-94cd-ae44caf91c31 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:08.379570 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e43b83a-4533-4493-94cd-ae44caf91c31 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:08.380020 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e43b83a-4533-4493-94cd-ae44caf91c31 None admin] PATH_INFO: `/v3/services` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:08.381432 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e43b83a-4533-4493-94cd-ae44caf91c31 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:08.381933 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e43b83a-4533-4493-94cd-ae44caf91c31 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:08.384119 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e43b83a-4533-4493-94cd-ae44caf91c31 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:08.392303 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 407/821] 10.222.0.55 () {56 vars in 1124 bytes} [Wed Aug 24 08:35:08 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:08.393898 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e43b83a-4533-4493-94cd-ae44caf91c31 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:08.443715 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:08.488991 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:08.521921 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95deb5ce-1e5d-466f-b734-bda25fadc87f None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:08.558281 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e43b83a-4533-4493-94cd-ae44caf91c31 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:08.570897 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb534d9b-0efe-44a6-8629-c744bc78c02d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:08.571729 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb534d9b-0efe-44a6-8629-c744bc78c02d None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:08.572135 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb534d9b-0efe-44a6-8629-c744bc78c02d None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:08.572600 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb534d9b-0efe-44a6-8629-c744bc78c02d None admin] PATH_INFO: `/v3/projects/service` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:08.582101 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb534d9b-0efe-44a6-8629-c744bc78c02d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:08.583999 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb534d9b-0efe-44a6-8629-c744bc78c02d None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:08.599840 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-cb534d9b-0efe-44a6-8629-c744bc78c02d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:35:08.599840 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:08.599840 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:08.599840 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:08.599840 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:08.599840 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:08.599840 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:08.599840 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:08.599840 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:08.599840 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:08.599840 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:08.599840 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:08.599840 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:35:08.599840 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:35:08.599840 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:35:08.601902 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:35:08.601902 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:08.601902 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:08.601902 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:08.601902 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:08.601902 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:08.601902 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:08.601902 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:08.601902 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:08.601902 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:08.601902 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:08.601902 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:08.601902 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:08.601902 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:08.601902 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:08.601902 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:08.601902 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:08.604274 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:35:08.604274 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:35:08.604274 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:35:08.604274 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:35:08.604274 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:35:08.604274 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:35:08.604274 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:35:08.604274 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:35:08.604274 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 415/822] 10.222.0.55 () {56 vars in 1111 bytes} [Wed Aug 24 08:35:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:08.604274 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb534d9b-0efe-44a6-8629-c744bc78c02d None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:08.629741 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e662f0-a7a1-4a89-a042-bc9140be52e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:08.630940 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e662f0-a7a1-4a89-a042-bc9140be52e1 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:08.631383 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e662f0-a7a1-4a89-a042-bc9140be52e1 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:08.631647 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e662f0-a7a1-4a89-a042-bc9140be52e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:08.736498 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e662f0-a7a1-4a89-a042-bc9140be52e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:08.738134 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e662f0-a7a1-4a89-a042-bc9140be52e1 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:08.771212 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb534d9b-0efe-44a6-8629-c744bc78c02d None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:08.807004 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:08.817850 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:08.839106 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b341dda-9923-40f6-8ba0-66b2cc85f347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:08.840173 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b341dda-9923-40f6-8ba0-66b2cc85f347 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:08.840700 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b341dda-9923-40f6-8ba0-66b2cc85f347 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:08.841889 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b341dda-9923-40f6-8ba0-66b2cc85f347 None admin] PATH_INFO: `/v3/services` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:08.842954 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b341dda-9923-40f6-8ba0-66b2cc85f347 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:08.843490 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b341dda-9923-40f6-8ba0-66b2cc85f347 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:08.846133 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b341dda-9923-40f6-8ba0-66b2cc85f347 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:08.863455 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 416/823] 10.222.0.55 () {56 vars in 1124 bytes} [Wed Aug 24 08:35:08 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:08.863455 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b341dda-9923-40f6-8ba0-66b2cc85f347 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:08.875598 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 408/824] 10.222.0.55 () {58 vars in 1282 bytes} [Wed Aug 24 08:35:08 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 08:35:08.880664 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e662f0-a7a1-4a89-a042-bc9140be52e1 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:09.013514 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e662f0-a7a1-4a89-a042-bc9140be52e1 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:09.037246 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b341dda-9923-40f6-8ba0-66b2cc85f347 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:09.063855 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb7fa78-5e53-4768-9043-f0392eef76ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:09.064835 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb7fa78-5e53-4768-9043-f0392eef76ce None admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:09.065298 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb7fa78-5e53-4768-9043-f0392eef76ce None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:09.065754 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb7fa78-5e53-4768-9043-f0392eef76ce None admin] PATH_INFO: `/v3/endpoints` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:09.066790 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb7fa78-5e53-4768-9043-f0392eef76ce None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:09.069064 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb7fa78-5e53-4768-9043-f0392eef76ce None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:09.095364 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d274ec-35d4-44be-a85f-1b4d6951b061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:09.096113 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d274ec-35d4-44be-a85f-1b4d6951b061 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:09.096659 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d274ec-35d4-44be-a85f-1b4d6951b061 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:09.101024 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d274ec-35d4-44be-a85f-1b4d6951b061 None admin] PATH_INFO: `/v3/projects` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:09.101024 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d274ec-35d4-44be-a85f-1b4d6951b061 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:09.101024 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d274ec-35d4-44be-a85f-1b4d6951b061 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:09.101024 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d274ec-35d4-44be-a85f-1b4d6951b061 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:09.114719 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 417/825] 10.222.0.55 () {56 vars in 1148 bytes} [Wed Aug 24 08:35:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:09.115723 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d274ec-35d4-44be-a85f-1b4d6951b061 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:09.165991 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:09.204997 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 409/826] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 08:35:08 2022] POST /identity/v3/endpoints => generated 346 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:35:09.206316 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb7fa78-5e53-4768-9043-f0392eef76ce None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:09.262018 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:09.278101 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d274ec-35d4-44be-a85f-1b4d6951b061 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:09.362852 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7d9254-a076-47cb-aeee-0683cb27cf97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:09.364102 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7d9254-a076-47cb-aeee-0683cb27cf97 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:09.364625 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7d9254-a076-47cb-aeee-0683cb27cf97 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:09.365120 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7d9254-a076-47cb-aeee-0683cb27cf97 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:09.387191 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7d9254-a076-47cb-aeee-0683cb27cf97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:09.390564 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7d9254-a076-47cb-aeee-0683cb27cf97 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:09.426001 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-8f7d9254-a076-47cb-aeee-0683cb27cf97 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:09.426001 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:09.426001 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:09.426001 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:09.426001 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:09.426001 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:09.426001 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:09.426001 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:09.426001 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:09.426001 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:09.426001 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:09.426001 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:09.426001 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:35:09.426001 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:35:09.426001 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:35:09.426001 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:35:09.427362 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:09.427362 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:09.427362 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:09.427362 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:09.427362 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:09.427362 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:09.427362 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:09.427362 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:09.427362 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:09.427362 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:09.427362 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:09.427362 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:09.427362 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:09.427362 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:09.427362 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:09.427362 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:09.427362 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:35:09.428373 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:35:09.428373 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:09.428373 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:35:09.431802 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 418/827] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:35:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:09.439879 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7d9254-a076-47cb-aeee-0683cb27cf97 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:09.452584 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb7fa78-5e53-4768-9043-f0392eef76ce None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:09.524874 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1ccf9d-6be7-494d-b318-00fd4dfdeb9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:09.525963 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1ccf9d-6be7-494d-b318-00fd4dfdeb9b None admin] REQUEST_METHOD: `PUT` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:09.526384 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1ccf9d-6be7-494d-b318-00fd4dfdeb9b None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:09.526825 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1ccf9d-6be7-494d-b318-00fd4dfdeb9b None admin] PATH_INFO: `/v3/projects/420b18725f2342159646fb86438318fd/users/f652a368b1cc42ae8b6f6c7a6dff6f2f/roles/bab90e02eebc473e9b911c8660ed9b76` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:09.558891 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1ccf9d-6be7-494d-b318-00fd4dfdeb9b None admin] RBAC: Authorizing `identity:create_grant(project_id=420b18725f2342159646fb86438318fd, user_id=f652a368b1cc42ae8b6f6c7a6dff6f2f, role_id=bab90e02eebc473e9b911c8660ed9b76)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:09.562522 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1ccf9d-6be7-494d-b318-00fd4dfdeb9b None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:09.622263 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:09.639569 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7d9254-a076-47cb-aeee-0683cb27cf97 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:09.651693 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 410/828] 10.222.0.55 () {58 vars in 1439 bytes} [Wed Aug 24 08:35:09 2022] PUT /identity/v3/projects/420b18725f2342159646fb86438318fd/users/f652a368b1cc42ae8b6f6c7a6dff6f2f/roles/bab90e02eebc473e9b911c8660ed9b76 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:35:09.715213 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69853d7e-3ffe-4a24-b6c1-c1e3c212161b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:09.717443 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69853d7e-3ffe-4a24-b6c1-c1e3c212161b None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:09.718219 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69853d7e-3ffe-4a24-b6c1-c1e3c212161b None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:09.718960 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69853d7e-3ffe-4a24-b6c1-c1e3c212161b None admin] PATH_INFO: `/v3/domains` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:09.720394 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69853d7e-3ffe-4a24-b6c1-c1e3c212161b None admin] RBAC: Adding query filter params (name=Default) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:09.721328 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69853d7e-3ffe-4a24-b6c1-c1e3c212161b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:09.724288 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69853d7e-3ffe-4a24-b6c1-c1e3c212161b None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:09.750842 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 419/829] 10.222.0.55 () {56 vars in 1109 bytes} [Wed Aug 24 08:35:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:35:09.759778 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1ccf9d-6be7-494d-b318-00fd4dfdeb9b None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:09.893452 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:09.933525 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1ccf9d-6be7-494d-b318-00fd4dfdeb9b None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:09.972252 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df04af4f-5d78-400b-87f8-87ceeaecefb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:09.973306 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df04af4f-5d78-400b-87f8-87ceeaecefb5 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:09.973798 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df04af4f-5d78-400b-87f8-87ceeaecefb5 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:09.974274 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df04af4f-5d78-400b-87f8-87ceeaecefb5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:09.983600 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df04af4f-5d78-400b-87f8-87ceeaecefb5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:09.985381 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df04af4f-5d78-400b-87f8-87ceeaecefb5 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:09.997709 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-df04af4f-5d78-400b-87f8-87ceeaecefb5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:35:09.997709 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:09.997709 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:09.997709 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:09.997709 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:09.997709 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:09.997709 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:09.997709 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:09.997709 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:09.997709 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:09.997709 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:09.997709 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:09.997709 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:35:09.997709 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:35:09.997709 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:35:09.998677 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:35:09.998677 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:09.998677 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:09.998677 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:09.998677 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:09.998677 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:09.998677 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:09.998677 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:09.998677 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:09.998677 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:09.998677 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:09.998677 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:09.998677 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:09.998677 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:09.998677 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:09.998677 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:09.998677 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:09.999708 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:35:09.999708 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:35:09.999708 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:35:09.999708 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:35:09.999708 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:35:09.999708 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:35:09.999708 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:35:09.999708 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:35:10.002176 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 411/830] 10.222.0.55 () {56 vars in 1111 bytes} [Wed Aug 24 08:35:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:10.011857 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69853d7e-3ffe-4a24-b6c1-c1e3c212161b None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:10.127507 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69853d7e-3ffe-4a24-b6c1-c1e3c212161b None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:10.167747 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb22134-b062-4eb0-b403-2160301cf74c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:10.168731 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb22134-b062-4eb0-b403-2160301cf74c None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:10.169273 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb22134-b062-4eb0-b403-2160301cf74c None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:10.169769 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb22134-b062-4eb0-b403-2160301cf74c None admin] PATH_INFO: `/v3/projects` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:10.170674 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb22134-b062-4eb0-b403-2160301cf74c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:10.171177 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb22134-b062-4eb0-b403-2160301cf74c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:10.172932 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb22134-b062-4eb0-b403-2160301cf74c None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:10.187165 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 420/831] 10.222.0.55 () {56 vars in 1148 bytes} [Wed Aug 24 08:35:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:35:10.213576 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df04af4f-5d78-400b-87f8-87ceeaecefb5 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:10.410046 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df04af4f-5d78-400b-87f8-87ceeaecefb5 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:10.469362 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:10.475890 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe92455-9a62-49ac-b79c-2a63f9b4e4bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:10.476945 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe92455-9a62-49ac-b79c-2a63f9b4e4bf None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:10.477414 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe92455-9a62-49ac-b79c-2a63f9b4e4bf None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:10.477898 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe92455-9a62-49ac-b79c-2a63f9b4e4bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:10.479250 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe92455-9a62-49ac-b79c-2a63f9b4e4bf None admin] RBAC: Adding query filter params (role.id=b870cd1acaa744e1ae1d58b9ac283821, scope.project.id=420b18725f2342159646fb86438318fd, user.id=f17da357e555420799ee681bffbcc268) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:10.479778 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe92455-9a62-49ac-b79c-2a63f9b4e4bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:10.482532 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe92455-9a62-49ac-b79c-2a63f9b4e4bf None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:10.495117 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 412/832] 10.222.0.55 () {56 vars in 1374 bytes} [Wed Aug 24 08:35:10 2022] GET /identity/v3/role_assignments?user.id=f17da357e555420799ee681bffbcc268&scope.project.id=420b18725f2342159646fb86438318fd&role.id=b870cd1acaa744e1ae1d58b9ac283821 => generated 608 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:35:10.920672 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270777d5-ff7f-42bf-9676-48ebaf52f777 None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:10.921485 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270777d5-ff7f-42bf-9676-48ebaf52f777 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:10.921485 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270777d5-ff7f-42bf-9676-48ebaf52f777 None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:10.922828 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 421/833] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:35:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:35:10.928751 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb608ba-20ab-45c5-8b6c-9d9867446f9b None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:10.929038 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb608ba-20ab-45c5-8b6c-9d9867446f9b None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:10.929314 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb608ba-20ab-45c5-8b6c-9d9867446f9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:10.950228 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-0bb608ba-20ab-45c5-8b6c-9d9867446f9b None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:35:10.951433 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-0bb608ba-20ab-45c5-8b6c-9d9867446f9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:35:11.035487 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 413/834] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:35:10 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 08:35:11.321543 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270777d5-ff7f-42bf-9676-48ebaf52f777 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:11.346428 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:11.531952 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270777d5-ff7f-42bf-9676-48ebaf52f777 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:11.600421 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6899dbca-9e66-483e-af63-8edbbc3a3a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:11.601527 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6899dbca-9e66-483e-af63-8edbbc3a3a27 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:11.602069 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6899dbca-9e66-483e-af63-8edbbc3a3a27 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:11.602553 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6899dbca-9e66-483e-af63-8edbbc3a3a27 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:11.624208 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6899dbca-9e66-483e-af63-8edbbc3a3a27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:35:11.633891 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-6899dbca-9e66-483e-af63-8edbbc3a3a27 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:35:11.633891 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:11.633891 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:11.633891 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:11.633891 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:11.633891 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:11.633891 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:11.633891 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:11.633891 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:11.633891 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:11.633891 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:11.633891 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:11.633891 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:35:11.633891 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:35:11.633891 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:35:11.633891 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:35:11.634881 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:35:11.634881 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:35:11.634881 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:35:11.634881 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:35:11.634881 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:11.634881 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:11.634881 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:11.634881 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:11.634881 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:11.634881 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:11.634881 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:11.634881 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:11.634881 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:11.634881 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:11.634881 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:11.634881 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:11.635721 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:11.635721 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:11.635721 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:11.635721 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:11.635721 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:35:11.635721 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:35:11.635721 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:35:11.635721 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:35:11.635721 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:35:11.635721 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:35:11.635721 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:35:11.635721 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:35:11.640876 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 422/835] 10.222.0.55 () {56 vars in 1096 bytes} [Wed Aug 24 08:35:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:11.646813 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb608ba-20ab-45c5-8b6c-9d9867446f9b None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:11.658147 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4432e339-ab52-46b8-8b7a-7f6fa6eb2b3b None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:11.658637 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4432e339-ab52-46b8-8b7a-7f6fa6eb2b3b None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:11.659779 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4432e339-ab52-46b8-8b7a-7f6fa6eb2b3b None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:11.662837 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 423/836] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:35:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:35:11.668840 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973f8e4f-eae1-483d-85d0-21629a713453 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:11.669357 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973f8e4f-eae1-483d-85d0-21629a713453 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:11.669826 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973f8e4f-eae1-483d-85d0-21629a713453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:11.704764 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-973f8e4f-eae1-483d-85d0-21629a713453 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:35:11.706128 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-973f8e4f-eae1-483d-85d0-21629a713453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:35:11.818078 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb608ba-20ab-45c5-8b6c-9d9867446f9b None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:11.847787 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:11.861284 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483f5c6b-e9be-4c3a-b2e0-432f8ae717de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:11.862647 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483f5c6b-e9be-4c3a-b2e0-432f8ae717de None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:11.862647 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483f5c6b-e9be-4c3a-b2e0-432f8ae717de None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:11.862647 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483f5c6b-e9be-4c3a-b2e0-432f8ae717de None admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:11.863315 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483f5c6b-e9be-4c3a-b2e0-432f8ae717de None admin] RBAC: Adding query filter params (name=admin) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:11.863622 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483f5c6b-e9be-4c3a-b2e0-432f8ae717de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:11.865085 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483f5c6b-e9be-4c3a-b2e0-432f8ae717de None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:11.875292 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 414/837] 10.222.0.55 () {56 vars in 1099 bytes} [Wed Aug 24 08:35:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:11.883705 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483f5c6b-e9be-4c3a-b2e0-432f8ae717de None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:11.926675 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:11.936614 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 424/838] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:35:11 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:35:12.050063 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483f5c6b-e9be-4c3a-b2e0-432f8ae717de None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:12.109982 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef04e915-3940-4f6c-a915-d4e71e6357e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:12.111402 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef04e915-3940-4f6c-a915-d4e71e6357e8 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:12.112122 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef04e915-3940-4f6c-a915-d4e71e6357e8 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:12.112860 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef04e915-3940-4f6c-a915-d4e71e6357e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:12.129875 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef04e915-3940-4f6c-a915-d4e71e6357e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:12.133198 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef04e915-3940-4f6c-a915-d4e71e6357e8 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:12.158065 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-ef04e915-3940-4f6c-a915-d4e71e6357e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:12.158065 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:12.158065 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:12.158065 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:12.158065 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:12.158065 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:12.158065 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:12.158065 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:12.158065 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:12.158065 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:12.158065 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:12.158065 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:12.158065 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:35:12.158065 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:35:12.158065 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:35:12.158065 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:35:12.160324 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:12.160324 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:12.160324 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:12.160324 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:12.160324 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:12.160324 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:12.160324 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:12.160324 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:12.160324 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:12.160324 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:12.160324 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:12.160324 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:12.160324 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:12.160324 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:12.160324 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:12.160324 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:12.160324 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:35:12.162158 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:35:12.162158 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:12.162158 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:35:12.162158 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 415/839] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:35:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:35:12.173694 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973f8e4f-eae1-483d-85d0-21629a713453 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:12.212342 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef04e915-3940-4f6c-a915-d4e71e6357e8 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:12.261284 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:12.327002 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef04e915-3940-4f6c-a915-d4e71e6357e8 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:12.364922 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2511f2-d856-48cd-b02b-1572ddf8354b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:12.365495 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2511f2-d856-48cd-b02b-1572ddf8354b None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:12.365782 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2511f2-d856-48cd-b02b-1572ddf8354b None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:12.366060 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2511f2-d856-48cd-b02b-1572ddf8354b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:12.366661 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2511f2-d856-48cd-b02b-1572ddf8354b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:12.368061 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2511f2-d856-48cd-b02b-1572ddf8354b None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:12.372879 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973f8e4f-eae1-483d-85d0-21629a713453 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:12.379051 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-ba2511f2-d856-48cd-b02b-1572ddf8354b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 08:35:12.379051 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:12.379051 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 08:35:12.379051 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 08:35:12.379051 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 08:35:12.379051 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 08:35:12.379051 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 08:35:12.379051 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:35:12.379051 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 08:35:12.379051 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:35:12.379051 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:35:12.379051 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:35:12.379051 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:12.379051 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:12.379051 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:12.379051 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:12.379051 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:12.380077 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:12.380077 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:12.380077 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:12.380077 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:12.380077 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:12.380077 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:12.380077 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 08:35:12.380077 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 08:35:12.380077 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 08:35:12.380077 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 08:35:12.380077 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:12.380077 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:12.380077 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:12.380077 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:12.380077 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:12.380077 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:12.381208 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:12.381208 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:12.381208 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:12.381208 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:12.381208 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:12.381208 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:12.381208 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:12.381208 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:12.381208 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:12.381208 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:12.381208 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 08:35:12.381208 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:35:12.381208 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 08:35:12.381208 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:35:12.381208 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 416/840] 10.222.0.55 () {56 vars in 1114 bytes} [Wed Aug 24 08:35:12 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:12.389055 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2511f2-d856-48cd-b02b-1572ddf8354b None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:12.424260 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad159785-4d30-4300-adbb-cd9dc25c1b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:12.425157 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad159785-4d30-4300-adbb-cd9dc25c1b24 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:12.425684 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad159785-4d30-4300-adbb-cd9dc25c1b24 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:12.426095 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad159785-4d30-4300-adbb-cd9dc25c1b24 None admin] PATH_INFO: `/v3/domains` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:12.426896 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad159785-4d30-4300-adbb-cd9dc25c1b24 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:12.427337 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad159785-4d30-4300-adbb-cd9dc25c1b24 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:12.428880 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad159785-4d30-4300-adbb-cd9dc25c1b24 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:12.442023 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 425/841] 10.222.0.55 () {56 vars in 1109 bytes} [Wed Aug 24 08:35:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:12.443556 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68c92f7-d376-4a4b-8479-ebd2fd33c21e None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:12.443971 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68c92f7-d376-4a4b-8479-ebd2fd33c21e None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:12.444364 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68c92f7-d376-4a4b-8479-ebd2fd33c21e None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:12.446188 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 426/842] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:35:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:35:12.447424 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68c92f7-d376-4a4b-8479-ebd2fd33c21e None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:12.521632 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:12.525288 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2511f2-d856-48cd-b02b-1572ddf8354b None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:12.526235 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:12.565373 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491933b8-ff7d-4723-bc10-a59b8f0801a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:12.566953 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491933b8-ff7d-4723-bc10-a59b8f0801a7 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:12.567233 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491933b8-ff7d-4723-bc10-a59b8f0801a7 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:12.567505 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491933b8-ff7d-4723-bc10-a59b8f0801a7 None admin] PATH_INFO: `/v3/services` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:12.568436 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491933b8-ff7d-4723-bc10-a59b8f0801a7 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:12.568764 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491933b8-ff7d-4723-bc10-a59b8f0801a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:12.570306 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491933b8-ff7d-4723-bc10-a59b8f0801a7 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:12.576896 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 417/843] 10.222.0.55 () {56 vars in 1114 bytes} [Wed Aug 24 08:35:12 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:35:12.578630 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297bd6bb-970b-488a-a360-6682950cf130 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:12.578959 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297bd6bb-970b-488a-a360-6682950cf130 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:12.579302 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297bd6bb-970b-488a-a360-6682950cf130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:12.598846 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-297bd6bb-970b-488a-a360-6682950cf130 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:35:12.600067 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-297bd6bb-970b-488a-a360-6682950cf130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:35:12.605721 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68c92f7-d376-4a4b-8479-ebd2fd33c21e None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:12.666847 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ec5c03-6b86-41d5-ae09-fed3a7d0bcdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:12.668023 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ec5c03-6b86-41d5-ae09-fed3a7d0bcdb None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:12.668764 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ec5c03-6b86-41d5-ae09-fed3a7d0bcdb None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:12.669345 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ec5c03-6b86-41d5-ae09-fed3a7d0bcdb None admin] PATH_INFO: `/v3/users/swift` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:12.680605 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 418/844] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:35:12 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:35:12.680605 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297bd6bb-970b-488a-a360-6682950cf130 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:12.691379 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ec5c03-6b86-41d5-ae09-fed3a7d0bcdb None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:12.694310 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ec5c03-6b86-41d5-ae09-fed3a7d0bcdb None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:12.717470 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-29ec5c03-6b86-41d5-ae09-fed3a7d0bcdb None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 08:35:12.717470 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:12.717470 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:12.717470 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:12.717470 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:12.717470 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:12.717470 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:12.717470 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:12.717470 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:12.717470 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:12.717470 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:12.717470 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:12.717470 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:35:12.717470 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:35:12.717470 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:35:12.717470 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:35:12.719533 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:12.719533 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:12.719533 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:35:12.719533 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:35:12.719533 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:35:12.719533 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:35:12.719533 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:12.719533 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:12.719533 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:12.719533 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:12.719533 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:12.719533 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:12.719533 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:12.719533 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:12.719533 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:12.719533 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:12.719533 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:12.719533 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:12.721649 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:12.721649 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:12.721649 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:35:12.721649 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:35:12.721649 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:35:12.721649 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:35:12.721649 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:35:12.721649 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:35:12.721649 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 08:35:12.721649 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:35:12.725277 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 427/845] 10.222.0.55 () {56 vars in 1096 bytes} [Wed Aug 24 08:35:12 2022] GET /identity/v3/users/swift => generated 83 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:12.734114 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ec5c03-6b86-41d5-ae09-fed3a7d0bcdb None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:12.801218 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:12.808346 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297bd6bb-970b-488a-a360-6682950cf130 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:12.848913 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76843954-c3bc-4624-a1e2-dfde162b4091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:12.849870 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76843954-c3bc-4624-a1e2-dfde162b4091 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:12.850330 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76843954-c3bc-4624-a1e2-dfde162b4091 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:12.850796 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76843954-c3bc-4624-a1e2-dfde162b4091 None admin] PATH_INFO: `/v3/services` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:12.851699 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76843954-c3bc-4624-a1e2-dfde162b4091 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:12.852167 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76843954-c3bc-4624-a1e2-dfde162b4091 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:12.853738 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76843954-c3bc-4624-a1e2-dfde162b4091 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:12.859997 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 419/846] 10.222.0.55 () {56 vars in 1114 bytes} [Wed Aug 24 08:35:12 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:12.861382 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607a5350-edc1-4530-8717-18cf890bfe80 None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:12.861669 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607a5350-edc1-4530-8717-18cf890bfe80 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:12.861923 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607a5350-edc1-4530-8717-18cf890bfe80 None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:12.863550 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 420/847] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:35:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:35:12.871000 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56d6a17-98ff-4032-b095-7fc9729a506e None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:12.871471 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56d6a17-98ff-4032-b095-7fc9729a506e None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:12.871949 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56d6a17-98ff-4032-b095-7fc9729a506e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:12.903986 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-f56d6a17-98ff-4032-b095-7fc9729a506e None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:35:12.909116 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-f56d6a17-98ff-4032-b095-7fc9729a506e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:35:12.940668 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ec5c03-6b86-41d5-ae09-fed3a7d0bcdb None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:12.971773 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:12.991830 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2057f03-95b5-402c-83f1-02f444400141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:12.992877 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2057f03-95b5-402c-83f1-02f444400141 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:12.993501 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2057f03-95b5-402c-83f1-02f444400141 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:12.994777 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2057f03-95b5-402c-83f1-02f444400141 None admin] PATH_INFO: `/v3/users` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:12.995812 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2057f03-95b5-402c-83f1-02f444400141 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:12.996333 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2057f03-95b5-402c-83f1-02f444400141 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:12.999217 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2057f03-95b5-402c-83f1-02f444400141 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:13.034434 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 428/848] 10.222.0.55 () {56 vars in 1135 bytes} [Wed Aug 24 08:35:12 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:13.035181 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 421/849] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:35:12 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:35:13.041828 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2057f03-95b5-402c-83f1-02f444400141 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:13.145842 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56d6a17-98ff-4032-b095-7fc9729a506e None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:13.174943 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2057f03-95b5-402c-83f1-02f444400141 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:13.249072 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aca38a8-b9b4-4e4e-891d-3b7d3c49eeed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:13.250140 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aca38a8-b9b4-4e4e-891d-3b7d3c49eeed None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:13.250599 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aca38a8-b9b4-4e4e-891d-3b7d3c49eeed None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:13.251054 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aca38a8-b9b4-4e4e-891d-3b7d3c49eeed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:13.275605 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:13.294817 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56d6a17-98ff-4032-b095-7fc9729a506e None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:13.355099 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:13.357972 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783f5848-51c9-4dba-9e2c-78021c964ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:13.358656 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783f5848-51c9-4dba-9e2c-78021c964ae9 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:13.358942 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783f5848-51c9-4dba-9e2c-78021c964ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:13.359320 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783f5848-51c9-4dba-9e2c-78021c964ae9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:13.378294 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783f5848-51c9-4dba-9e2c-78021c964ae9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:35:13.385218 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-783f5848-51c9-4dba-9e2c-78021c964ae9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:35:13.385218 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:13.385218 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:13.385218 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:13.385218 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:13.385218 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:13.385218 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:13.385218 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:13.385218 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:13.385218 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:13.385218 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:13.385218 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:13.385218 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:35:13.385218 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:35:13.385218 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:35:13.385218 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:35:13.393534 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:35:13.393534 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:35:13.393534 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:35:13.393534 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:35:13.393534 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:13.393534 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:13.393534 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:13.393534 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:13.393534 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:13.393534 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:13.393534 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:13.393534 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:13.393534 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:13.393534 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:13.393534 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:13.393534 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:13.395132 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:13.395132 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:13.395132 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:13.395132 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:13.395132 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:35:13.395132 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:35:13.395132 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:35:13.395132 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:35:13.395132 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:35:13.395132 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:35:13.395132 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:35:13.395132 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:35:13.395132 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 422/850] 10.222.0.55 () {56 vars in 1102 bytes} [Wed Aug 24 08:35:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:35:13.401195 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783f5848-51c9-4dba-9e2c-78021c964ae9 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:13.409281 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aca38a8-b9b4-4e4e-891d-3b7d3c49eeed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:13.410761 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aca38a8-b9b4-4e4e-891d-3b7d3c49eeed None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:13.523125 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 429/851] 10.222.0.55 () {58 vars in 1282 bytes} [Wed Aug 24 08:35:13 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 08:35:13.523794 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aca38a8-b9b4-4e4e-891d-3b7d3c49eeed None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:13.573296 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783f5848-51c9-4dba-9e2c-78021c964ae9 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:13.615349 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f1449c-f8fe-4fcf-a2c1-2ab5e7594a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:13.616266 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f1449c-f8fe-4fcf-a2c1-2ab5e7594a15 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:13.616706 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f1449c-f8fe-4fcf-a2c1-2ab5e7594a15 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:13.617128 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f1449c-f8fe-4fcf-a2c1-2ab5e7594a15 None admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:13.618098 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f1449c-f8fe-4fcf-a2c1-2ab5e7594a15 None admin] RBAC: Adding query filter params (name=service) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:13.618590 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f1449c-f8fe-4fcf-a2c1-2ab5e7594a15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:13.620833 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f1449c-f8fe-4fcf-a2c1-2ab5e7594a15 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:13.631469 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:13.639710 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 423/852] 10.222.0.55 () {56 vars in 1103 bytes} [Wed Aug 24 08:35:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:35:13.639710 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f1449c-f8fe-4fcf-a2c1-2ab5e7594a15 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:13.652343 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:13.665050 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aca38a8-b9b4-4e4e-891d-3b7d3c49eeed None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:13.718243 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841607d2-a859-41b1-abaf-3c9765e5ae99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:13.719465 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841607d2-a859-41b1-abaf-3c9765e5ae99 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:13.720053 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841607d2-a859-41b1-abaf-3c9765e5ae99 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:13.720583 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841607d2-a859-41b1-abaf-3c9765e5ae99 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:13.721623 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841607d2-a859-41b1-abaf-3c9765e5ae99 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:13.723697 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841607d2-a859-41b1-abaf-3c9765e5ae99 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:13.735140 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-841607d2-a859-41b1-abaf-3c9765e5ae99 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 08:35:13.735140 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:13.735140 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 08:35:13.735140 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 08:35:13.735140 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 08:35:13.735140 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 08:35:13.735140 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 08:35:13.735140 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:35:13.735140 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 08:35:13.735140 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:35:13.735140 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:35:13.735140 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:35:13.735140 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:13.735140 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:13.735140 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:13.735140 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:13.736318 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:13.736318 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:13.736318 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:13.736318 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:13.736318 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:13.736318 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:13.736318 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:13.736318 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 08:35:13.736318 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 08:35:13.736318 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 08:35:13.736318 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 08:35:13.736318 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:13.736318 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:13.736318 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:13.736318 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:13.736318 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:13.736318 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:13.738199 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:13.738199 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:13.738199 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:13.738199 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:13.738199 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:13.738199 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:13.738199 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:13.738199 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:13.738199 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:13.738199 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:13.738199 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 08:35:13.738199 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:35:13.738199 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 08:35:13.738199 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:35:13.740807 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 430/853] 10.222.0.55 () {56 vars in 1132 bytes} [Wed Aug 24 08:35:13 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:13.742740 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841607d2-a859-41b1-abaf-3c9765e5ae99 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:13.769657 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f1449c-f8fe-4fcf-a2c1-2ab5e7594a15 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:13.813132 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4580fd-9e00-416d-8ed0-1fdaf63ce90a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:13.814110 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4580fd-9e00-416d-8ed0-1fdaf63ce90a None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:13.814512 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4580fd-9e00-416d-8ed0-1fdaf63ce90a None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:13.814854 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4580fd-9e00-416d-8ed0-1fdaf63ce90a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:13.829371 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4580fd-9e00-416d-8ed0-1fdaf63ce90a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:13.831102 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4580fd-9e00-416d-8ed0-1fdaf63ce90a None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:13.847079 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-8f4580fd-9e00-416d-8ed0-1fdaf63ce90a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:13.847079 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:13.847079 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:13.847079 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:13.847079 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:13.847079 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:13.847079 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:13.847079 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:13.847079 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:13.847079 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:13.847079 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:13.847079 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:13.847079 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:35:13.847079 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:35:13.847079 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:35:13.847079 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:35:13.848251 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:13.848251 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:13.848251 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:13.848251 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:13.848251 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:13.848251 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:13.848251 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:13.848251 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:13.848251 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:13.848251 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:13.848251 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:13.848251 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:13.848251 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:13.848251 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:13.848251 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:13.848251 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:13.848251 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:35:13.849334 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:35:13.849334 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:13.849334 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:35:13.850772 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 424/854] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:35:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:13.851809 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4580fd-9e00-416d-8ed0-1fdaf63ce90a None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:13.888038 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841607d2-a859-41b1-abaf-3c9765e5ae99 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:13.931039 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:13.935772 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27329ae4-a710-458a-9a79-ba5ab8cd4462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:13.936694 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27329ae4-a710-458a-9a79-ba5ab8cd4462 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:13.937165 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27329ae4-a710-458a-9a79-ba5ab8cd4462 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:13.937560 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27329ae4-a710-458a-9a79-ba5ab8cd4462 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:13.954131 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27329ae4-a710-458a-9a79-ba5ab8cd4462 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:13.954131 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27329ae4-a710-458a-9a79-ba5ab8cd4462 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:13.955375 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:13.968404 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-27329ae4-a710-458a-9a79-ba5ab8cd4462 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:13.968404 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:13.968404 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:13.968404 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:13.968404 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:13.968404 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:13.968404 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:13.968404 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:13.968404 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:13.968404 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:13.968404 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:13.968404 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:13.968404 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:35:13.968404 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:35:13.968404 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:35:13.968404 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:35:13.969472 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:13.969472 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:13.969472 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:13.969472 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:13.969472 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:13.969472 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:13.969472 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:13.969472 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:13.969472 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:13.969472 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:13.969472 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:13.969472 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:13.969472 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:13.969472 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:13.969472 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:13.969472 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:13.969472 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:35:13.970370 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:35:13.970370 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:13.970370 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:35:13.970370 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 431/855] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:35:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:13.971089 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27329ae4-a710-458a-9a79-ba5ab8cd4462 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:13.994084 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4580fd-9e00-416d-8ed0-1fdaf63ce90a None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:14.036473 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba6a34e-a051-4695-aa72-5464f6648b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:14.037157 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba6a34e-a051-4695-aa72-5464f6648b38 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:14.037433 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba6a34e-a051-4695-aa72-5464f6648b38 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:14.037735 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba6a34e-a051-4695-aa72-5464f6648b38 None admin] PATH_INFO: `/v3/services` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:14.038346 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba6a34e-a051-4695-aa72-5464f6648b38 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:14.038644 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba6a34e-a051-4695-aa72-5464f6648b38 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:14.040028 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba6a34e-a051-4695-aa72-5464f6648b38 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:14.056182 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 425/856] 10.222.0.55 () {56 vars in 1126 bytes} [Wed Aug 24 08:35:13 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:14.056182 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba6a34e-a051-4695-aa72-5464f6648b38 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:14.112528 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27329ae4-a710-458a-9a79-ba5ab8cd4462 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:14.160445 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8730f7-e51e-4998-bdf6-f2a97caf1b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:14.161092 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8730f7-e51e-4998-bdf6-f2a97caf1b13 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:14.161650 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8730f7-e51e-4998-bdf6-f2a97caf1b13 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:14.161927 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8730f7-e51e-4998-bdf6-f2a97caf1b13 None admin] PATH_INFO: `/v3/domains` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:14.162581 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8730f7-e51e-4998-bdf6-f2a97caf1b13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:14.162878 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8730f7-e51e-4998-bdf6-f2a97caf1b13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:14.165178 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8730f7-e51e-4998-bdf6-f2a97caf1b13 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:14.174522 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba6a34e-a051-4695-aa72-5464f6648b38 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:14.185453 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 432/857] 10.222.0.55 () {56 vars in 1109 bytes} [Wed Aug 24 08:35:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:14.186998 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8730f7-e51e-4998-bdf6-f2a97caf1b13 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:14.221032 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:14.234113 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f40d98d-7e4e-4195-9ebd-c9b28a74318f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:14.234918 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f40d98d-7e4e-4195-9ebd-c9b28a74318f None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:14.235326 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f40d98d-7e4e-4195-9ebd-c9b28a74318f None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:14.235674 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f40d98d-7e4e-4195-9ebd-c9b28a74318f None admin] PATH_INFO: `/v3/domains` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:14.236524 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f40d98d-7e4e-4195-9ebd-c9b28a74318f None admin] RBAC: Adding query filter params (name=Default) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:14.237018 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f40d98d-7e4e-4195-9ebd-c9b28a74318f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:14.238670 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f40d98d-7e4e-4195-9ebd-c9b28a74318f None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:14.254511 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 426/858] 10.222.0.55 () {56 vars in 1109 bytes} [Wed Aug 24 08:35:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:14.256609 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f40d98d-7e4e-4195-9ebd-c9b28a74318f None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:14.269685 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:14.323048 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8730f7-e51e-4998-bdf6-f2a97caf1b13 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:14.368966 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58052adc-69df-4ab6-8655-f42a156cb778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:14.369521 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58052adc-69df-4ab6-8655-f42a156cb778 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:14.369795 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58052adc-69df-4ab6-8655-f42a156cb778 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:14.370061 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58052adc-69df-4ab6-8655-f42a156cb778 None admin] PATH_INFO: `/v3/services` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:14.370710 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58052adc-69df-4ab6-8655-f42a156cb778 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:14.371007 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58052adc-69df-4ab6-8655-f42a156cb778 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:14.372429 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58052adc-69df-4ab6-8655-f42a156cb778 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:14.385118 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 433/859] 10.222.0.55 () {56 vars in 1126 bytes} [Wed Aug 24 08:35:14 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:14.385118 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58052adc-69df-4ab6-8655-f42a156cb778 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:14.412127 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f40d98d-7e4e-4195-9ebd-c9b28a74318f None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:14.486672 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6813859a-a154-45cf-a8a5-d62902093855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:14.487984 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6813859a-a154-45cf-a8a5-d62902093855 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:14.488654 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6813859a-a154-45cf-a8a5-d62902093855 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:14.489705 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6813859a-a154-45cf-a8a5-d62902093855 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:14.504984 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6813859a-a154-45cf-a8a5-d62902093855 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:14.508401 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6813859a-a154-45cf-a8a5-d62902093855 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:14.534266 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-6813859a-a154-45cf-a8a5-d62902093855 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:35:14.534266 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:14.534266 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:14.534266 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:14.534266 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:14.534266 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:14.534266 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:14.534266 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:14.534266 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:14.534266 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:14.534266 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:14.534266 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:14.534266 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:35:14.534266 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:35:14.534266 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:35:14.536236 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:35:14.536236 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:14.536236 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:14.536236 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:14.536236 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:14.536236 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:14.536236 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:14.536236 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:14.536236 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:14.536236 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:14.536236 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:14.536236 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:14.536236 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:14.536236 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:14.536236 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:14.536236 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:14.536236 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:14.538297 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:35:14.538297 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:35:14.538297 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:35:14.538297 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:35:14.538297 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:35:14.538297 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:35:14.538297 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:35:14.538297 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:35:14.541850 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 427/860] 10.222.0.55 () {56 vars in 1111 bytes} [Wed Aug 24 08:35:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:14.549370 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6813859a-a154-45cf-a8a5-d62902093855 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:14.593092 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58052adc-69df-4ab6-8655-f42a156cb778 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:14.635299 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:14.656972 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:14.665320 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79442bf-4da1-4d67-afe1-18e6ff899aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:14.666774 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79442bf-4da1-4d67-afe1-18e6ff899aab None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:14.667498 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79442bf-4da1-4d67-afe1-18e6ff899aab None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:14.668241 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79442bf-4da1-4d67-afe1-18e6ff899aab None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:14.683375 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79442bf-4da1-4d67-afe1-18e6ff899aab None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:14.686506 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79442bf-4da1-4d67-afe1-18e6ff899aab None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:14.709762 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-d79442bf-4da1-4d67-afe1-18e6ff899aab None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 08:35:14.709762 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:14.709762 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:14.709762 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:14.709762 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:14.709762 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:14.709762 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:14.709762 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:14.709762 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:14.709762 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:14.709762 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:14.709762 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:14.709762 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:35:14.709762 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:35:14.709762 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:35:14.709762 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:35:14.711592 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:14.711592 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:14.711592 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:35:14.711592 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:35:14.711592 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:35:14.711592 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:35:14.711592 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:14.711592 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:14.711592 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:14.711592 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:14.711592 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:14.711592 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:14.711592 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:14.711592 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:14.711592 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:14.711592 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:14.711592 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:14.711592 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:14.713746 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:14.713746 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:14.713746 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:35:14.713746 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:35:14.713746 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:35:14.713746 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:35:14.713746 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:35:14.713746 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:35:14.713746 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 08:35:14.713746 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:35:14.717981 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 434/861] 10.222.0.55 () {56 vars in 1117 bytes} [Wed Aug 24 08:35:14 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:14.726619 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79442bf-4da1-4d67-afe1-18e6ff899aab None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:14.738966 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6813859a-a154-45cf-a8a5-d62902093855 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:14.807708 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b80046-814e-49f3-a326-6f6b16bf7c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:14.808710 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b80046-814e-49f3-a326-6f6b16bf7c31 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:14.809389 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b80046-814e-49f3-a326-6f6b16bf7c31 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:14.809908 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b80046-814e-49f3-a326-6f6b16bf7c31 None admin] PATH_INFO: `/v3/projects` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:14.811935 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b80046-814e-49f3-a326-6f6b16bf7c31 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:14.812472 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b80046-814e-49f3-a326-6f6b16bf7c31 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:14.816067 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b80046-814e-49f3-a326-6f6b16bf7c31 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:14.847926 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 428/862] 10.222.0.55 () {56 vars in 1148 bytes} [Wed Aug 24 08:35:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:14.854525 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b80046-814e-49f3-a326-6f6b16bf7c31 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:14.862277 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79442bf-4da1-4d67-afe1-18e6ff899aab None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:14.908532 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4429ffec-f0ca-4def-a2fb-257683b18ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:14.909133 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4429ffec-f0ca-4def-a2fb-257683b18ee5 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:14.909391 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4429ffec-f0ca-4def-a2fb-257683b18ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:14.909709 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4429ffec-f0ca-4def-a2fb-257683b18ee5 None admin] PATH_INFO: `/v3/users` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:14.910461 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4429ffec-f0ca-4def-a2fb-257683b18ee5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:14.916583 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4429ffec-f0ca-4def-a2fb-257683b18ee5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:14.916583 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4429ffec-f0ca-4def-a2fb-257683b18ee5 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:14.947184 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 435/863] 10.222.0.55 () {56 vars in 1149 bytes} [Wed Aug 24 08:35:14 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:14.953360 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4429ffec-f0ca-4def-a2fb-257683b18ee5 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:14.998208 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:15.026565 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b80046-814e-49f3-a326-6f6b16bf7c31 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:15.036108 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:15.085253 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dd5497-232d-4043-a0cc-86645350e880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:15.085506 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4429ffec-f0ca-4def-a2fb-257683b18ee5 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:15.086250 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dd5497-232d-4043-a0cc-86645350e880 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:15.086707 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dd5497-232d-4043-a0cc-86645350e880 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:15.087181 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dd5497-232d-4043-a0cc-86645350e880 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:15.088454 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dd5497-232d-4043-a0cc-86645350e880 None admin] RBAC: Adding query filter params (role.id=b870cd1acaa744e1ae1d58b9ac283821, scope.project.id=420b18725f2342159646fb86438318fd, user.id=a0ebb5ccabef45378f0c1da69721e051) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:15.089013 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dd5497-232d-4043-a0cc-86645350e880 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:15.091709 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dd5497-232d-4043-a0cc-86645350e880 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:15.112577 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 429/864] 10.222.0.55 () {56 vars in 1374 bytes} [Wed Aug 24 08:35:14 2022] GET /identity/v3/role_assignments?user.id=a0ebb5ccabef45378f0c1da69721e051&scope.project.id=420b18725f2342159646fb86438318fd&role.id=b870cd1acaa744e1ae1d58b9ac283821 => generated 608 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:35:15.112577 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a794f8-5866-4af0-b903-82fe95149c49 None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:15.112577 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a794f8-5866-4af0-b903-82fe95149c49 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:15.112577 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a794f8-5866-4af0-b903-82fe95149c49 None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:15.112577 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 430/865] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:35:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:35:15.120830 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e134d4d6-8698-41dd-ab07-65dd73f32e46 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:15.121392 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e134d4d6-8698-41dd-ab07-65dd73f32e46 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:15.121888 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e134d4d6-8698-41dd-ab07-65dd73f32e46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:15.152850 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c90dad-ad50-4cb4-a7ae-da7764cc1c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:15.153845 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-e134d4d6-8698-41dd-ab07-65dd73f32e46 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:35:15.154521 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c90dad-ad50-4cb4-a7ae-da7764cc1c69 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:15.155215 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c90dad-ad50-4cb4-a7ae-da7764cc1c69 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:15.155954 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c90dad-ad50-4cb4-a7ae-da7764cc1c69 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:15.156129 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-e134d4d6-8698-41dd-ab07-65dd73f32e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:35:15.274971 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c90dad-ad50-4cb4-a7ae-da7764cc1c69 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:15.276843 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c90dad-ad50-4cb4-a7ae-da7764cc1c69 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:15.295364 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 431/866] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:35:15 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:35:15.330321 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:15.417530 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 436/867] 10.222.0.55 () {58 vars in 1282 bytes} [Wed Aug 24 08:35:14 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 08:35:15.426313 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e134d4d6-8698-41dd-ab07-65dd73f32e46 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:15.596940 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c90dad-ad50-4cb4-a7ae-da7764cc1c69 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:15.598121 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:15.601958 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e134d4d6-8698-41dd-ab07-65dd73f32e46 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:15.644044 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f20844-99d4-4f9a-b23c-832145b2b5c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:15.644740 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f20844-99d4-4f9a-b23c-832145b2b5c6 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:15.645021 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f20844-99d4-4f9a-b23c-832145b2b5c6 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:15.645299 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f20844-99d4-4f9a-b23c-832145b2b5c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:15.657227 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f20844-99d4-4f9a-b23c-832145b2b5c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:15.658850 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f20844-99d4-4f9a-b23c-832145b2b5c6 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:15.676721 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-17f20844-99d4-4f9a-b23c-832145b2b5c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:15.676721 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:15.676721 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:15.676721 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:15.676721 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:15.676721 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:15.676721 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:15.676721 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:15.676721 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:15.676721 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:15.676721 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:15.676721 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:15.676721 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:35:15.676721 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:35:15.676721 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:35:15.676721 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:35:15.678772 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:15.678772 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:15.678772 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:15.678772 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:15.678772 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:15.678772 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:15.678772 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:15.678772 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:15.678772 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:15.678772 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:15.678772 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:15.678772 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:15.678772 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:15.678772 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:15.678772 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:15.678772 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:15.678772 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:35:15.680481 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:35:15.680481 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:15.680481 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:35:15.680481 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 432/868] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:35:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:15.691149 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f20844-99d4-4f9a-b23c-832145b2b5c6 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:15.727231 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c90dad-ad50-4cb4-a7ae-da7764cc1c69 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:15.774040 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd14479-eaaf-4b83-847a-1d38073e3454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:15.774663 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd14479-eaaf-4b83-847a-1d38073e3454 None admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:15.774935 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd14479-eaaf-4b83-847a-1d38073e3454 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:15.775187 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd14479-eaaf-4b83-847a-1d38073e3454 None admin] PATH_INFO: `/v3/services` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:15.775861 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd14479-eaaf-4b83-847a-1d38073e3454 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:15.777520 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd14479-eaaf-4b83-847a-1d38073e3454 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:15.784759 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:15.881728 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f20844-99d4-4f9a-b23c-832145b2b5c6 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:15.897065 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 437/869] 10.222.0.55 () {60 vars in 1141 bytes} [Wed Aug 24 08:35:15 2022] POST /identity/v3/services => generated 251 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:35:15.963408 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dfe546-7829-4b8f-a955-e29d9b3f167f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:15.964432 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dfe546-7829-4b8f-a955-e29d9b3f167f None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:15.964946 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dfe546-7829-4b8f-a955-e29d9b3f167f None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:15.965419 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dfe546-7829-4b8f-a955-e29d9b3f167f None admin] PATH_INFO: `/v3/domains` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:15.967237 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dfe546-7829-4b8f-a955-e29d9b3f167f None admin] RBAC: Adding query filter params (name=Default) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:15.967793 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dfe546-7829-4b8f-a955-e29d9b3f167f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:15.970521 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dfe546-7829-4b8f-a955-e29d9b3f167f None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:15.994849 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 433/870] 10.222.0.55 () {56 vars in 1109 bytes} [Wed Aug 24 08:35:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:16.003345 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd14479-eaaf-4b83-847a-1d38073e3454 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:16.195530 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd14479-eaaf-4b83-847a-1d38073e3454 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:16.255044 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0c2b67-07ea-47d3-8cad-2943b9790116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:16.256449 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0c2b67-07ea-47d3-8cad-2943b9790116 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:16.256970 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0c2b67-07ea-47d3-8cad-2943b9790116 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:16.257456 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0c2b67-07ea-47d3-8cad-2943b9790116 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:16.271121 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0c2b67-07ea-47d3-8cad-2943b9790116 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:16.273863 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0c2b67-07ea-47d3-8cad-2943b9790116 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:16.296756 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-aa0c2b67-07ea-47d3-8cad-2943b9790116 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:35:16.296756 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:16.296756 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:16.296756 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:16.296756 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:16.296756 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:16.296756 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:16.296756 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:16.296756 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:16.296756 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:16.296756 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:16.296756 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:16.296756 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:35:16.296756 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:35:16.296756 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:35:16.297747 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:35:16.297747 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:16.297747 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:16.297747 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:16.297747 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:16.297747 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:16.297747 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:16.297747 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:16.297747 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:16.297747 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:16.297747 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:16.297747 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:16.297747 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:16.297747 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:16.297747 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:16.297747 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:16.297747 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:16.298646 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:35:16.298646 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:35:16.298646 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:35:16.298646 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:35:16.298646 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:35:16.298646 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:35:16.298646 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:35:16.298646 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:35:16.303819 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 438/871] 10.222.0.55 () {56 vars in 1111 bytes} [Wed Aug 24 08:35:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:16.308616 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dfe546-7829-4b8f-a955-e29d9b3f167f None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:16.327934 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:16.428089 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dfe546-7829-4b8f-a955-e29d9b3f167f None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:16.468832 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25843bc-7b3f-48e7-b36f-171c56c9596b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:16.469756 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25843bc-7b3f-48e7-b36f-171c56c9596b None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:16.470212 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25843bc-7b3f-48e7-b36f-171c56c9596b None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:16.470680 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25843bc-7b3f-48e7-b36f-171c56c9596b None admin] PATH_INFO: `/v3/projects` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:16.471572 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25843bc-7b3f-48e7-b36f-171c56c9596b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:16.472071 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25843bc-7b3f-48e7-b36f-171c56c9596b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:16.474036 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25843bc-7b3f-48e7-b36f-171c56c9596b None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:16.487788 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 434/872] 10.222.0.55 () {56 vars in 1148 bytes} [Wed Aug 24 08:35:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:16.497901 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0c2b67-07ea-47d3-8cad-2943b9790116 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:16.572957 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:16.617252 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0c2b67-07ea-47d3-8cad-2943b9790116 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:16.656688 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4a9583-a90c-4d50-bfc0-0ff7ced69372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:16.657738 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4a9583-a90c-4d50-bfc0-0ff7ced69372 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:16.658026 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4a9583-a90c-4d50-bfc0-0ff7ced69372 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:16.658301 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4a9583-a90c-4d50-bfc0-0ff7ced69372 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:16.659040 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4a9583-a90c-4d50-bfc0-0ff7ced69372 None admin] RBAC: Adding query filter params (role.id=bab90e02eebc473e9b911c8660ed9b76, scope.project.id=420b18725f2342159646fb86438318fd, user.id=f652a368b1cc42ae8b6f6c7a6dff6f2f) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:16.659353 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4a9583-a90c-4d50-bfc0-0ff7ced69372 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:16.660829 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4a9583-a90c-4d50-bfc0-0ff7ced69372 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:16.667632 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 439/873] 10.222.0.55 () {56 vars in 1374 bytes} [Wed Aug 24 08:35:16 2022] GET /identity/v3/role_assignments?user.id=f652a368b1cc42ae8b6f6c7a6dff6f2f&scope.project.id=420b18725f2342159646fb86438318fd&role.id=bab90e02eebc473e9b911c8660ed9b76 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:16.732654 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a028e9e-111a-44af-84e7-6808e25873f3 None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:16.733301 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a028e9e-111a-44af-84e7-6808e25873f3 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:16.733803 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a028e9e-111a-44af-84e7-6808e25873f3 None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:16.735866 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 435/874] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:35:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:35:16.742135 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd90b85-b1aa-43e7-bc16-7bbf1dd51636 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:16.742785 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd90b85-b1aa-43e7-bc16-7bbf1dd51636 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:16.743263 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd90b85-b1aa-43e7-bc16-7bbf1dd51636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:16.765254 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-4dd90b85-b1aa-43e7-bc16-7bbf1dd51636 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:35:16.766910 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-4dd90b85-b1aa-43e7-bc16-7bbf1dd51636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:35:16.851299 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 440/875] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:35:16 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 08:35:17.128670 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a028e9e-111a-44af-84e7-6808e25873f3 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:17.207227 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:17.248820 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a028e9e-111a-44af-84e7-6808e25873f3 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:17.287514 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74977c14-d1a8-4a80-9952-54cfad309507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:17.288133 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74977c14-d1a8-4a80-9952-54cfad309507 None admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:17.288419 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74977c14-d1a8-4a80-9952-54cfad309507 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:17.288729 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74977c14-d1a8-4a80-9952-54cfad309507 None admin] PATH_INFO: `/v3/services` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:17.289395 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74977c14-d1a8-4a80-9952-54cfad309507 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:17.290769 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74977c14-d1a8-4a80-9952-54cfad309507 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:17.355895 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 436/876] 10.222.0.55 () {60 vars in 1141 bytes} [Wed Aug 24 08:35:17 2022] POST /identity/v3/services => generated 269 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:35:18.056062 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b5d432-4286-4f8a-ae2f-a3974fd83044 None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:18.057124 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b5d432-4286-4f8a-ae2f-a3974fd83044 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:18.057623 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b5d432-4286-4f8a-ae2f-a3974fd83044 None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:18.059547 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 441/877] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:35:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:35:18.065500 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0961f075-0f54-47b2-992a-9406c2d11cea None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:18.066111 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0961f075-0f54-47b2-992a-9406c2d11cea None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:18.066583 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0961f075-0f54-47b2-992a-9406c2d11cea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:18.085787 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-0961f075-0f54-47b2-992a-9406c2d11cea None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:35:18.087332 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-0961f075-0f54-47b2-992a-9406c2d11cea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:35:18.167490 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 437/878] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:35:18 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:35:18.422589 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673219a8-d5bd-4acc-83e2-01f9e0744fcd None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:18.423582 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673219a8-d5bd-4acc-83e2-01f9e0744fcd None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:18.424339 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673219a8-d5bd-4acc-83e2-01f9e0744fcd None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:18.427450 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 442/879] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:35:18 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:35:18.436642 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b37629-57bc-4133-a125-61a974163802 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:18.437132 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b37629-57bc-4133-a125-61a974163802 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:18.437610 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b37629-57bc-4133-a125-61a974163802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:18.438384 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673219a8-d5bd-4acc-83e2-01f9e0744fcd None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:18.470704 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-26b37629-57bc-4133-a125-61a974163802 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:35:18.474349 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-26b37629-57bc-4133-a125-61a974163802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:35:18.518097 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:18.563625 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:18.594338 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 438/880] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:35:18 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:35:18.609148 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673219a8-d5bd-4acc-83e2-01f9e0744fcd None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:18.653255 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e63d3c4-4a9b-454a-9451-f9fd20c27663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:18.654905 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63d3c4-4a9b-454a-9451-f9fd20c27663 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:18.655415 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63d3c4-4a9b-454a-9451-f9fd20c27663 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:18.655891 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63d3c4-4a9b-454a-9451-f9fd20c27663 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:18.656724 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e63d3c4-4a9b-454a-9451-f9fd20c27663 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:18.658331 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e63d3c4-4a9b-454a-9451-f9fd20c27663 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:18.668487 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-8e63d3c4-4a9b-454a-9451-f9fd20c27663 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 08:35:18.668487 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:18.668487 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 08:35:18.668487 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 08:35:18.668487 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 08:35:18.668487 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 08:35:18.668487 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 08:35:18.668487 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:35:18.668487 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 08:35:18.668487 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:35:18.668487 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:35:18.668487 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:35:18.668487 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:18.668487 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:18.668487 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:18.668487 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:18.668487 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:18.669577 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:18.669577 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:18.669577 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:18.669577 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:18.669577 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:18.669577 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:18.669577 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 08:35:18.669577 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 08:35:18.669577 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 08:35:18.669577 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 08:35:18.669577 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:18.669577 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:18.669577 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:18.669577 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:18.669577 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:18.669577 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:18.670430 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:18.670430 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:18.670430 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:18.670430 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:18.670430 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:18.670430 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:18.670430 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:18.670430 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:18.670430 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:18.670430 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:18.670430 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 08:35:18.670430 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:35:18.670430 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 08:35:18.670430 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:35:18.673360 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 443/881] 10.222.0.55 () {56 vars in 1114 bytes} [Wed Aug 24 08:35:18 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:35:18.680315 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b37629-57bc-4133-a125-61a974163802 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:18.792707 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b37629-57bc-4133-a125-61a974163802 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:18.830656 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645a22dc-ce9f-413b-8e2c-246fe5e7c437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:18.831632 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645a22dc-ce9f-413b-8e2c-246fe5e7c437 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:18.832119 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645a22dc-ce9f-413b-8e2c-246fe5e7c437 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:18.832625 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645a22dc-ce9f-413b-8e2c-246fe5e7c437 None admin] PATH_INFO: `/v3/services` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:18.833577 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645a22dc-ce9f-413b-8e2c-246fe5e7c437 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:18.834089 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645a22dc-ce9f-413b-8e2c-246fe5e7c437 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:18.835676 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645a22dc-ce9f-413b-8e2c-246fe5e7c437 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:18.841971 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 439/882] 10.222.0.55 () {56 vars in 1114 bytes} [Wed Aug 24 08:35:18 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:35:18.848672 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e63d3c4-4a9b-454a-9451-f9fd20c27663 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:19.003370 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e63d3c4-4a9b-454a-9451-f9fd20c27663 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:19.019485 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:19.042603 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7bac82-0085-4d81-85db-065eda50e3e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:19.043577 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7bac82-0085-4d81-85db-065eda50e3e8 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:19.044089 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7bac82-0085-4d81-85db-065eda50e3e8 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:19.045158 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7bac82-0085-4d81-85db-065eda50e3e8 None admin] PATH_INFO: `/v3/services` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:19.046001 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7bac82-0085-4d81-85db-065eda50e3e8 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:19.046521 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7bac82-0085-4d81-85db-065eda50e3e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:19.048108 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7bac82-0085-4d81-85db-065eda50e3e8 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:19.054136 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 444/883] 10.222.0.55 () {56 vars in 1114 bytes} [Wed Aug 24 08:35:18 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:35:19.060472 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645a22dc-ce9f-413b-8e2c-246fe5e7c437 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:19.096444 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7bac82-0085-4d81-85db-065eda50e3e8 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:19.148996 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:19.245949 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7bac82-0085-4d81-85db-065eda50e3e8 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:19.270738 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645a22dc-ce9f-413b-8e2c-246fe5e7c437 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:19.294166 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345bc38d-f7b9-41f0-a207-c72f1ad36c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:19.294884 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345bc38d-f7b9-41f0-a207-c72f1ad36c74 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:19.295166 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345bc38d-f7b9-41f0-a207-c72f1ad36c74 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:19.295495 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345bc38d-f7b9-41f0-a207-c72f1ad36c74 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:19.296118 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345bc38d-f7b9-41f0-a207-c72f1ad36c74 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:19.297868 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345bc38d-f7b9-41f0-a207-c72f1ad36c74 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:19.309429 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-345bc38d-f7b9-41f0-a207-c72f1ad36c74 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 08:35:19.309429 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:19.309429 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 08:35:19.309429 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 08:35:19.309429 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 08:35:19.309429 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 08:35:19.309429 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 08:35:19.309429 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:35:19.309429 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 08:35:19.309429 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:35:19.309429 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:35:19.309429 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:35:19.309429 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:19.309429 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:19.309429 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:19.309429 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:19.309429 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:19.311402 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:19.311402 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:19.311402 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:19.311402 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:19.311402 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:19.311402 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:19.311402 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 08:35:19.311402 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 08:35:19.311402 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 08:35:19.311402 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 08:35:19.311402 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:19.311402 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:19.311402 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:19.311402 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:19.311402 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:19.311402 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:19.313049 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:19.313049 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:19.313049 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:19.313049 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:19.313049 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:19.313049 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:19.313049 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:19.313049 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:19.313049 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:19.313049 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:19.313049 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 08:35:19.313049 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:35:19.313049 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 08:35:19.313049 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:35:19.313049 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 445/884] 10.222.0.55 () {56 vars in 1126 bytes} [Wed Aug 24 08:35:19 2022] GET /identity/v3/services/object-store => generated 93 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:35:19.323350 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345bc38d-f7b9-41f0-a207-c72f1ad36c74 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:19.335442 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a76164f-6c28-4a38-b5ad-b7a53b52abe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:19.336069 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a76164f-6c28-4a38-b5ad-b7a53b52abe1 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:19.336348 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a76164f-6c28-4a38-b5ad-b7a53b52abe1 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:19.336664 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a76164f-6c28-4a38-b5ad-b7a53b52abe1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:19.337389 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a76164f-6c28-4a38-b5ad-b7a53b52abe1 None admin] RBAC: Adding query filter params (interface=public, service_id=d52159154ba1499eaba1e4416f9b0c1d, region_id=RegionOne) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:19.337697 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a76164f-6c28-4a38-b5ad-b7a53b52abe1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:19.339086 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a76164f-6c28-4a38-b5ad-b7a53b52abe1 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:19.352659 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 440/885] 10.222.0.55 () {56 vars in 1251 bytes} [Wed Aug 24 08:35:19 2022] GET /identity/v3/endpoints?service_id=d52159154ba1499eaba1e4416f9b0c1d&interface=public®ion_id=RegionOne => generated 194 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:19.355798 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a76164f-6c28-4a38-b5ad-b7a53b52abe1 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:19.424992 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:19.521838 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345bc38d-f7b9-41f0-a207-c72f1ad36c74 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:19.546953 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a76164f-6c28-4a38-b5ad-b7a53b52abe1 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:19.568785 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:19.573679 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f33f099-63ba-41b5-9d88-fc4472fc5997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:19.574472 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f33f099-63ba-41b5-9d88-fc4472fc5997 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:19.574634 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f33f099-63ba-41b5-9d88-fc4472fc5997 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:19.574927 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f33f099-63ba-41b5-9d88-fc4472fc5997 None admin] PATH_INFO: `/v3/services` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:19.576110 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f33f099-63ba-41b5-9d88-fc4472fc5997 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:19.576494 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f33f099-63ba-41b5-9d88-fc4472fc5997 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:19.578486 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f33f099-63ba-41b5-9d88-fc4472fc5997 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:19.587405 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 446/886] 10.222.0.55 () {56 vars in 1122 bytes} [Wed Aug 24 08:35:19 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:19.588452 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c613dd-a0ec-4b74-b4e6-7d88d74a942d None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:19.588789 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c613dd-a0ec-4b74-b4e6-7d88d74a942d None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:19.589070 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c613dd-a0ec-4b74-b4e6-7d88d74a942d None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:19.590578 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 447/887] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:35:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:35:19.596038 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c613dd-a0ec-4b74-b4e6-7d88d74a942d None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:19.604744 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f684f4c4-670c-4c17-892e-21b4a49bc4e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:19.605799 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f684f4c4-670c-4c17-892e-21b4a49bc4e3 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:19.606293 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f684f4c4-670c-4c17-892e-21b4a49bc4e3 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:19.606777 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f684f4c4-670c-4c17-892e-21b4a49bc4e3 None admin] PATH_INFO: `/v3/services` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:19.607854 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f684f4c4-670c-4c17-892e-21b4a49bc4e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:19.610543 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f684f4c4-670c-4c17-892e-21b4a49bc4e3 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:19.631692 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 441/888] 10.222.0.55 () {56 vars in 1087 bytes} [Wed Aug 24 08:35:19 2022] GET /identity/v3/services => generated 1268 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:35:19.631692 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221db8d6-ec79-48dc-8865-39ee565e4a97 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:19.631692 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221db8d6-ec79-48dc-8865-39ee565e4a97 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:19.631692 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221db8d6-ec79-48dc-8865-39ee565e4a97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:19.659952 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-221db8d6-ec79-48dc-8865-39ee565e4a97 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:35:19.662055 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-221db8d6-ec79-48dc-8865-39ee565e4a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:35:19.788672 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 442/889] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:35:19 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:35:19.807911 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c613dd-a0ec-4b74-b4e6-7d88d74a942d None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:19.823003 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b105c5-8893-4c51-9fca-ff7cf05ee7c4 None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:19.823491 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b105c5-8893-4c51-9fca-ff7cf05ee7c4 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:19.823964 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b105c5-8893-4c51-9fca-ff7cf05ee7c4 None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:19.824465 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:19.828042 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 443/890] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:35:19 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:35:19.835488 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badda3cd-2b79-4e9d-bf11-1ec66dfd3043 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:19.837605 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badda3cd-2b79-4e9d-bf11-1ec66dfd3043 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:19.838174 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badda3cd-2b79-4e9d-bf11-1ec66dfd3043 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:19.868111 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e925990-514b-4ca9-b15a-60735ec6688c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:19.869181 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e925990-514b-4ca9-b15a-60735ec6688c None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:19.869700 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e925990-514b-4ca9-b15a-60735ec6688c None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:19.870179 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e925990-514b-4ca9-b15a-60735ec6688c None admin] PATH_INFO: `/v3/services` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:19.871062 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e925990-514b-4ca9-b15a-60735ec6688c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:19.871880 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e925990-514b-4ca9-b15a-60735ec6688c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:19.874280 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e925990-514b-4ca9-b15a-60735ec6688c None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:19.874722 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-badda3cd-2b79-4e9d-bf11-1ec66dfd3043 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:35:19.877012 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-badda3cd-2b79-4e9d-bf11-1ec66dfd3043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:35:19.880995 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 448/891] 10.222.0.55 () {56 vars in 1122 bytes} [Wed Aug 24 08:35:19 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:19.983476 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 444/892] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:35:19 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:35:20.197778 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e925990-514b-4ca9-b15a-60735ec6688c None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:20.265622 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badda3cd-2b79-4e9d-bf11-1ec66dfd3043 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:20.321014 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:20.334445 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e925990-514b-4ca9-b15a-60735ec6688c None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:20.383331 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a4758d-1f69-4335-ac50-19b31bd2bfb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:20.383989 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a4758d-1f69-4335-ac50-19b31bd2bfb9 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:20.384267 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a4758d-1f69-4335-ac50-19b31bd2bfb9 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:20.384555 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a4758d-1f69-4335-ac50-19b31bd2bfb9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:20.399356 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a4758d-1f69-4335-ac50-19b31bd2bfb9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:35:20.404804 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-e5a4758d-1f69-4335-ac50-19b31bd2bfb9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 08:35:20.404804 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:20.404804 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:20.404804 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:20.404804 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:20.404804 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:20.404804 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:20.404804 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:20.404804 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:20.404804 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:20.404804 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:20.404804 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:20.404804 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:35:20.404804 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:35:20.404804 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:35:20.404804 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:35:20.405837 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:35:20.405837 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:35:20.405837 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:35:20.405837 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:35:20.405837 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:20.405837 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:20.405837 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:20.405837 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:20.405837 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:20.405837 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:20.405837 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:20.405837 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:20.405837 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:20.405837 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:20.405837 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:20.405837 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:20.406725 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:20.406725 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:20.406725 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:20.406725 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:20.406725 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:35:20.406725 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:35:20.406725 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:35:20.406725 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:35:20.406725 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:35:20.406725 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:35:20.406725 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 08:35:20.406725 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:35:20.407647 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 449/893] 10.222.0.55 () {56 vars in 1120 bytes} [Wed Aug 24 08:35:20 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:20.417219 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a4758d-1f69-4335-ac50-19b31bd2bfb9 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:20.461730 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badda3cd-2b79-4e9d-bf11-1ec66dfd3043 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:20.476653 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:20.534511 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a25888d-5619-4e8a-aed6-fe7072120a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:20.535999 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a25888d-5619-4e8a-aed6-fe7072120a06 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:20.536787 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a25888d-5619-4e8a-aed6-fe7072120a06 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:20.537486 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a25888d-5619-4e8a-aed6-fe7072120a06 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:20.538816 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a25888d-5619-4e8a-aed6-fe7072120a06 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:20.541952 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a25888d-5619-4e8a-aed6-fe7072120a06 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:20.560796 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-3a25888d-5619-4e8a-aed6-fe7072120a06 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 08:35:20.560796 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:20.560796 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 08:35:20.560796 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 08:35:20.560796 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 08:35:20.560796 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 08:35:20.560796 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 08:35:20.560796 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:35:20.560796 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 08:35:20.560796 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:35:20.560796 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:35:20.560796 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:35:20.560796 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:20.560796 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:20.560796 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:20.560796 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:20.562302 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:20.562302 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:20.562302 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:20.562302 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:20.562302 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:20.562302 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:20.562302 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:20.562302 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 08:35:20.562302 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 08:35:20.562302 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 08:35:20.562302 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 08:35:20.562302 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:20.562302 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:20.562302 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:20.562302 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:20.562302 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:20.562302 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:20.563862 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:20.563862 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:20.563862 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:20.563862 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:20.563862 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:20.563862 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:20.563862 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:20.563862 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:20.563862 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:20.563862 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:20.563862 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 08:35:20.563862 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:35:20.563862 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 08:35:20.563862 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:35:20.568008 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 445/894] 10.222.0.55 () {56 vars in 1132 bytes} [Wed Aug 24 08:35:20 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:20.575989 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a25888d-5619-4e8a-aed6-fe7072120a06 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:20.623585 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a4758d-1f69-4335-ac50-19b31bd2bfb9 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:20.701633 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:20.705593 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a25888d-5619-4e8a-aed6-fe7072120a06 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:20.716684 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207494b4-3d33-4bcf-b40f-9f746375051e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:20.716684 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207494b4-3d33-4bcf-b40f-9f746375051e None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:20.716684 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207494b4-3d33-4bcf-b40f-9f746375051e None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:20.718060 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207494b4-3d33-4bcf-b40f-9f746375051e None admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:20.719555 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207494b4-3d33-4bcf-b40f-9f746375051e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:20.720421 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207494b4-3d33-4bcf-b40f-9f746375051e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:20.723441 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207494b4-3d33-4bcf-b40f-9f746375051e None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:20.744650 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac227e40-8ac7-43fe-a405-a14139b09f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:20.744650 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac227e40-8ac7-43fe-a405-a14139b09f65 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:20.744650 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac227e40-8ac7-43fe-a405-a14139b09f65 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:20.744650 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac227e40-8ac7-43fe-a405-a14139b09f65 None admin] PATH_INFO: `/v3/services` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:20.745266 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac227e40-8ac7-43fe-a405-a14139b09f65 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:20.745266 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac227e40-8ac7-43fe-a405-a14139b09f65 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:20.746272 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 450/895] 10.222.0.55 () {56 vars in 1115 bytes} [Wed Aug 24 08:35:20 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:35:20.746440 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac227e40-8ac7-43fe-a405-a14139b09f65 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:20.748487 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207494b4-3d33-4bcf-b40f-9f746375051e None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:20.756826 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 446/896] 10.222.0.55 () {56 vars in 1126 bytes} [Wed Aug 24 08:35:20 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:20.758202 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac227e40-8ac7-43fe-a405-a14139b09f65 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:20.876175 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac227e40-8ac7-43fe-a405-a14139b09f65 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:20.915893 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52145691-0531-4283-bd23-285c7c23f720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:20.916572 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52145691-0531-4283-bd23-285c7c23f720 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:20.916860 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52145691-0531-4283-bd23-285c7c23f720 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:20.917161 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52145691-0531-4283-bd23-285c7c23f720 None admin] PATH_INFO: `/v3/services` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:20.917775 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52145691-0531-4283-bd23-285c7c23f720 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:20.918072 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52145691-0531-4283-bd23-285c7c23f720 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:20.919525 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52145691-0531-4283-bd23-285c7c23f720 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:20.930713 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:20.931020 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 447/897] 10.222.0.55 () {56 vars in 1126 bytes} [Wed Aug 24 08:35:20 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:20.933086 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52145691-0531-4283-bd23-285c7c23f720 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:20.947868 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207494b4-3d33-4bcf-b40f-9f746375051e None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:20.968295 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:21.018750 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7384c188-e633-4f9e-9229-781b9dcac3bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:21.020399 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7384c188-e633-4f9e-9229-781b9dcac3bf None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:21.021316 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7384c188-e633-4f9e-9229-781b9dcac3bf None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:21.022060 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7384c188-e633-4f9e-9229-781b9dcac3bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:21.043934 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7384c188-e633-4f9e-9229-781b9dcac3bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:21.047435 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7384c188-e633-4f9e-9229-781b9dcac3bf None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:21.052036 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52145691-0531-4283-bd23-285c7c23f720 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:21.080107 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-7384c188-e633-4f9e-9229-781b9dcac3bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:21.080107 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:21.080107 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:21.080107 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:21.080107 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:21.080107 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:21.080107 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:21.080107 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:21.080107 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:21.080107 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:21.080107 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:21.080107 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:21.080107 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:35:21.080107 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:35:21.080107 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:35:21.080107 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:35:21.082152 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:21.082152 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:21.082152 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:21.082152 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:21.082152 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:21.082152 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:21.082152 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:21.082152 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:21.082152 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:21.082152 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:21.082152 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:21.082152 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:21.082152 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:21.082152 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:21.082152 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:21.082152 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:21.082152 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:35:21.084193 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:35:21.084193 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:21.084193 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:35:21.092579 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d62ee42-e10b-4f94-af68-77464a5fd6ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:21.092579 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d62ee42-e10b-4f94-af68-77464a5fd6ca None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:21.092579 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d62ee42-e10b-4f94-af68-77464a5fd6ca None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:21.092579 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d62ee42-e10b-4f94-af68-77464a5fd6ca None admin] PATH_INFO: `/v3/endpoints` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:21.093750 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d62ee42-e10b-4f94-af68-77464a5fd6ca None admin] RBAC: Adding query filter params (interface=public, service_id=183fdc9167394be1b07774aef98306e3, region_id=RegionOne) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:21.093981 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 451/898] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:35:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:21.094266 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d62ee42-e10b-4f94-af68-77464a5fd6ca None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:21.095850 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d62ee42-e10b-4f94-af68-77464a5fd6ca None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:21.100047 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7384c188-e633-4f9e-9229-781b9dcac3bf None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:21.107562 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 448/899] 10.222.0.55 () {56 vars in 1251 bytes} [Wed Aug 24 08:35:20 2022] GET /identity/v3/endpoints?service_id=183fdc9167394be1b07774aef98306e3&interface=public®ion_id=RegionOne => generated 194 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:35:21.109139 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d62ee42-e10b-4f94-af68-77464a5fd6ca None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:21.221614 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:21.232156 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d62ee42-e10b-4f94-af68-77464a5fd6ca None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:21.275819 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a015e2-4f7d-4c36-a7f6-c48a84c6b78f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:21.276654 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a015e2-4f7d-4c36-a7f6-c48a84c6b78f None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:21.277074 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a015e2-4f7d-4c36-a7f6-c48a84c6b78f None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:21.277442 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a015e2-4f7d-4c36-a7f6-c48a84c6b78f None admin] PATH_INFO: `/v3/services` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:21.278203 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a015e2-4f7d-4c36-a7f6-c48a84c6b78f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:21.279972 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a015e2-4f7d-4c36-a7f6-c48a84c6b78f None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:21.292742 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 449/900] 10.222.0.55 () {56 vars in 1087 bytes} [Wed Aug 24 08:35:21 2022] GET /identity/v3/services => generated 1268 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:35:21.320772 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7384c188-e633-4f9e-9229-781b9dcac3bf None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:21.393304 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ee0b16-0c2c-4d56-898a-f1b0729742ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:21.394711 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ee0b16-0c2c-4d56-898a-f1b0729742ce None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:21.395404 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ee0b16-0c2c-4d56-898a-f1b0729742ce None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:21.396088 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ee0b16-0c2c-4d56-898a-f1b0729742ce None admin] PATH_INFO: `/v3/domains` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:21.397545 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ee0b16-0c2c-4d56-898a-f1b0729742ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:21.398349 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ee0b16-0c2c-4d56-898a-f1b0729742ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:21.401398 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ee0b16-0c2c-4d56-898a-f1b0729742ce None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:21.427614 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 452/901] 10.222.0.55 () {56 vars in 1109 bytes} [Wed Aug 24 08:35:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:21.438026 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a015e2-4f7d-4c36-a7f6-c48a84c6b78f None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:21.645789 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a015e2-4f7d-4c36-a7f6-c48a84c6b78f None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:21.714523 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3329be-3bee-4a71-a974-71463c939491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:21.716000 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3329be-3bee-4a71-a974-71463c939491 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:21.716809 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3329be-3bee-4a71-a974-71463c939491 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:21.717508 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3329be-3bee-4a71-a974-71463c939491 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:21.733089 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3329be-3bee-4a71-a974-71463c939491 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:21.736370 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3329be-3bee-4a71-a974-71463c939491 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:21.760877 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-7f3329be-3bee-4a71-a974-71463c939491 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 08:35:21.760877 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:21.760877 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:21.760877 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:21.760877 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:21.760877 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:21.760877 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:21.760877 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:21.760877 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:21.760877 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:21.760877 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:21.760877 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:21.760877 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:35:21.760877 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:35:21.760877 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:35:21.760877 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:35:21.763051 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:21.763051 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:21.763051 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:35:21.763051 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:35:21.763051 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:35:21.763051 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:35:21.763051 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:21.763051 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:21.763051 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:21.763051 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:21.763051 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:21.763051 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:21.763051 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:21.763051 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:21.763051 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:21.763051 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:21.763051 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:21.763051 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:21.765882 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:21.765882 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:21.765882 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:35:21.765882 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:35:21.765882 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:35:21.765882 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:35:21.765882 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:35:21.765882 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:35:21.765882 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 08:35:21.765882 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:35:21.769621 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 450/902] 10.222.0.55 () {56 vars in 1117 bytes} [Wed Aug 24 08:35:21 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:21.778951 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ee0b16-0c2c-4d56-898a-f1b0729742ce None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:21.786457 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:21.825332 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a493c6-d00e-4941-9b70-b94a46d9356f None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:21.826343 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a493c6-d00e-4941-9b70-b94a46d9356f None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:21.826502 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a493c6-d00e-4941-9b70-b94a46d9356f None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:21.828140 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 451/903] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:35:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:35:21.833960 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f61b958-741d-47c9-9195-5fa6835a933d None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:21.834244 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f61b958-741d-47c9-9195-5fa6835a933d None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:21.834552 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f61b958-741d-47c9-9195-5fa6835a933d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:21.857703 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-9f61b958-741d-47c9-9195-5fa6835a933d None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:35:21.859106 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-9f61b958-741d-47c9-9195-5fa6835a933d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:35:21.880760 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:21.932708 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ee0b16-0c2c-4d56-898a-f1b0729742ce None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:21.970754 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f013605-fe84-48e7-ad3d-0ef21ff11cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:21.971422 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f013605-fe84-48e7-ad3d-0ef21ff11cc2 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:21.971698 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f013605-fe84-48e7-ad3d-0ef21ff11cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:21.971971 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f013605-fe84-48e7-ad3d-0ef21ff11cc2 None admin] PATH_INFO: `/v3/users` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:21.972793 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f013605-fe84-48e7-ad3d-0ef21ff11cc2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:21.973090 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f013605-fe84-48e7-ad3d-0ef21ff11cc2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:21.974527 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f013605-fe84-48e7-ad3d-0ef21ff11cc2 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:21.990150 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 452/904] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:35:21 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 08:35:22.004718 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 453/905] 10.222.0.55 () {56 vars in 1149 bytes} [Wed Aug 24 08:35:21 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:22.015356 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f61b958-741d-47c9-9195-5fa6835a933d None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:22.211501 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f61b958-741d-47c9-9195-5fa6835a933d None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:22.261120 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f013605-fe84-48e7-ad3d-0ef21ff11cc2 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:22.282229 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc29357-94c3-4f61-a154-e18716c2c388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:22.283290 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc29357-94c3-4f61-a154-e18716c2c388 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:22.283727 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc29357-94c3-4f61-a154-e18716c2c388 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:22.284139 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc29357-94c3-4f61-a154-e18716c2c388 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:22.304879 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:22.371002 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc29357-94c3-4f61-a154-e18716c2c388 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:22.373148 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc29357-94c3-4f61-a154-e18716c2c388 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:22.417073 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:22.424274 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f013605-fe84-48e7-ad3d-0ef21ff11cc2 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:22.467199 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a572e0-bc43-45b4-9bb7-475bb38c7232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:22.467745 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a572e0-bc43-45b4-9bb7-475bb38c7232 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:22.468024 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a572e0-bc43-45b4-9bb7-475bb38c7232 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:22.468294 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a572e0-bc43-45b4-9bb7-475bb38c7232 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:22.469052 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a572e0-bc43-45b4-9bb7-475bb38c7232 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:22.470434 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a572e0-bc43-45b4-9bb7-475bb38c7232 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:22.479856 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-f9a572e0-bc43-45b4-9bb7-475bb38c7232 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 08:35:22.479856 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:22.479856 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 08:35:22.479856 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 08:35:22.479856 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 08:35:22.479856 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 08:35:22.479856 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 08:35:22.479856 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:35:22.479856 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 08:35:22.479856 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:35:22.479856 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:35:22.479856 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:35:22.479856 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:22.479856 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:22.479856 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:22.479856 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:22.479856 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:22.486561 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:22.486561 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:22.486561 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:22.486561 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:22.486561 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:22.486561 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:22.486561 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 08:35:22.486561 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 08:35:22.486561 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 08:35:22.486561 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 08:35:22.486561 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:22.486561 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:22.486561 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:22.486561 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:22.486561 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:22.486561 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:22.487609 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:22.487609 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:22.487609 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:22.487609 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:22.487609 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:22.487609 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:22.487609 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:22.487609 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:22.487609 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:22.487609 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:22.487609 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 08:35:22.487609 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:35:22.487609 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 08:35:22.487609 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:35:22.487609 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 454/906] 10.222.0.55 () {56 vars in 1114 bytes} [Wed Aug 24 08:35:22 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:22.488651 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3bd497-7dec-47e2-a263-f2492788a189 None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:22.488651 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3bd497-7dec-47e2-a263-f2492788a189 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:22.488651 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3bd497-7dec-47e2-a263-f2492788a189 None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:22.488651 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 455/907] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:35:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:35:22.490015 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3bd497-7dec-47e2-a263-f2492788a189 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:22.518157 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 453/908] 10.222.0.55 () {58 vars in 1282 bytes} [Wed Aug 24 08:35:22 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 08:35:22.518157 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee30aa0d-ab13-499f-a456-91c98e567aed None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:22.518157 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee30aa0d-ab13-499f-a456-91c98e567aed None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:22.518157 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee30aa0d-ab13-499f-a456-91c98e567aed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:22.536974 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-ee30aa0d-ab13-499f-a456-91c98e567aed None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:35:22.538208 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-ee30aa0d-ab13-499f-a456-91c98e567aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:35:22.590048 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:22.623486 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 454/909] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:35:22 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:35:22.624761 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee30aa0d-ab13-499f-a456-91c98e567aed None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:22.683327 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3bd497-7dec-47e2-a263-f2492788a189 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:22.717771 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:22.727612 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce7c904-d0fb-47aa-b255-171dfb27d506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:22.728979 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce7c904-d0fb-47aa-b255-171dfb27d506 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:22.729572 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce7c904-d0fb-47aa-b255-171dfb27d506 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:22.732693 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce7c904-d0fb-47aa-b255-171dfb27d506 None admin] PATH_INFO: `/v3/services` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:22.732693 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce7c904-d0fb-47aa-b255-171dfb27d506 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:22.732693 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce7c904-d0fb-47aa-b255-171dfb27d506 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:22.733408 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce7c904-d0fb-47aa-b255-171dfb27d506 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:22.744451 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 456/910] 10.222.0.55 () {56 vars in 1114 bytes} [Wed Aug 24 08:35:22 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:22.745596 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce7c904-d0fb-47aa-b255-171dfb27d506 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:22.820452 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee30aa0d-ab13-499f-a456-91c98e567aed None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:22.878143 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f4a6a0-a6ca-458e-a55d-2dd9a3ee7709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:22.878882 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f4a6a0-a6ca-458e-a55d-2dd9a3ee7709 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:22.879222 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f4a6a0-a6ca-458e-a55d-2dd9a3ee7709 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:22.879556 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f4a6a0-a6ca-458e-a55d-2dd9a3ee7709 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:22.893861 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f4a6a0-a6ca-458e-a55d-2dd9a3ee7709 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:22.894675 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce7c904-d0fb-47aa-b255-171dfb27d506 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:22.895548 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f4a6a0-a6ca-458e-a55d-2dd9a3ee7709 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:22.915221 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-d0f4a6a0-a6ca-458e-a55d-2dd9a3ee7709 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:22.915221 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:22.915221 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:22.915221 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:22.915221 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:22.915221 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:22.915221 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:22.915221 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:22.915221 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:22.915221 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:22.915221 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:22.915221 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:22.915221 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:35:22.915221 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:35:22.915221 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:35:22.915221 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:35:22.917295 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:22.917295 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:22.917295 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:22.917295 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:22.917295 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:22.917295 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:22.917295 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:22.917295 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:22.917295 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:22.917295 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:22.917295 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:22.917295 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:22.917295 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:22.917295 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:22.917295 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:22.917295 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:22.917295 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:35:22.919339 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:35:22.919339 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:22.919339 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:35:22.919339 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 455/911] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:35:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:22.919339 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f4a6a0-a6ca-458e-a55d-2dd9a3ee7709 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:22.953286 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:22.968470 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cdc7e8-22f7-44c2-9c01-68b5a242aa7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:22.969554 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cdc7e8-22f7-44c2-9c01-68b5a242aa7e None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:22.970062 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cdc7e8-22f7-44c2-9c01-68b5a242aa7e None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:22.970567 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cdc7e8-22f7-44c2-9c01-68b5a242aa7e None admin] PATH_INFO: `/v3/services` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:22.971745 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cdc7e8-22f7-44c2-9c01-68b5a242aa7e None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:22.972299 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cdc7e8-22f7-44c2-9c01-68b5a242aa7e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:22.974983 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cdc7e8-22f7-44c2-9c01-68b5a242aa7e None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:22.983995 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 457/912] 10.222.0.55 () {56 vars in 1114 bytes} [Wed Aug 24 08:35:22 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:22.985581 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cdc7e8-22f7-44c2-9c01-68b5a242aa7e None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:23.039279 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f4a6a0-a6ca-458e-a55d-2dd9a3ee7709 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:23.077862 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b265e2-5fd7-4383-b822-f89c0fe2e9e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:23.078831 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b265e2-5fd7-4383-b822-f89c0fe2e9e8 None admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:23.079421 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b265e2-5fd7-4383-b822-f89c0fe2e9e8 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:23.079690 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b265e2-5fd7-4383-b822-f89c0fe2e9e8 None admin] PATH_INFO: `/v3/services` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:23.080352 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b265e2-5fd7-4383-b822-f89c0fe2e9e8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:23.081770 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b265e2-5fd7-4383-b822-f89c0fe2e9e8 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:23.084269 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:23.158513 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 456/913] 10.222.0.55 () {60 vars in 1141 bytes} [Wed Aug 24 08:35:22 2022] POST /identity/v3/services => generated 241 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:35:23.158513 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b265e2-5fd7-4383-b822-f89c0fe2e9e8 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:23.179677 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cdc7e8-22f7-44c2-9c01-68b5a242aa7e None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:23.259446 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d24fcde-b68e-47a1-afd9-4896b8d12ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:23.260316 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d24fcde-b68e-47a1-afd9-4896b8d12ea0 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:23.260777 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d24fcde-b68e-47a1-afd9-4896b8d12ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:23.261296 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d24fcde-b68e-47a1-afd9-4896b8d12ea0 None admin] PATH_INFO: `/v3/domains` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:23.262117 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d24fcde-b68e-47a1-afd9-4896b8d12ea0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:23.262586 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d24fcde-b68e-47a1-afd9-4896b8d12ea0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:23.264823 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d24fcde-b68e-47a1-afd9-4896b8d12ea0 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:23.287232 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 458/914] 10.222.0.55 () {56 vars in 1109 bytes} [Wed Aug 24 08:35:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:23.288900 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d24fcde-b68e-47a1-afd9-4896b8d12ea0 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:23.315282 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:23.342720 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b265e2-5fd7-4383-b822-f89c0fe2e9e8 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:23.417311 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c6b2c6-de66-491e-b2b1-51e0206b0890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:23.418063 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c6b2c6-de66-491e-b2b1-51e0206b0890 None admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:23.418342 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c6b2c6-de66-491e-b2b1-51e0206b0890 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:23.418661 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c6b2c6-de66-491e-b2b1-51e0206b0890 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:23.419389 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c6b2c6-de66-491e-b2b1-51e0206b0890 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:23.421500 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c6b2c6-de66-491e-b2b1-51e0206b0890 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:23.437726 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d24fcde-b68e-47a1-afd9-4896b8d12ea0 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:23.451859 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:23.476834 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3f6334-b63e-4c0c-9ce0-b0cf66078370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:23.477480 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3f6334-b63e-4c0c-9ce0-b0cf66078370 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:23.477756 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3f6334-b63e-4c0c-9ce0-b0cf66078370 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:23.478026 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3f6334-b63e-4c0c-9ce0-b0cf66078370 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:23.486008 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3f6334-b63e-4c0c-9ce0-b0cf66078370 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:23.487553 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3f6334-b63e-4c0c-9ce0-b0cf66078370 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:23.500804 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-9a3f6334-b63e-4c0c-9ce0-b0cf66078370 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:35:23.500804 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:23.500804 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:23.500804 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:23.500804 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:23.500804 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:23.500804 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:23.500804 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:23.500804 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:23.500804 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:23.500804 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:23.500804 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:23.500804 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:35:23.500804 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:35:23.500804 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:35:23.503159 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:35:23.503159 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:23.503159 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:23.503159 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:23.503159 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:23.503159 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:23.503159 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:23.503159 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:23.503159 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:23.503159 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:23.503159 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:23.503159 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:23.503159 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:23.503159 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:23.503159 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:23.503159 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:23.503159 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:23.505737 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:35:23.505737 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:35:23.505737 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:35:23.505737 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:35:23.505737 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:35:23.505737 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:35:23.505737 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:35:23.505737 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:35:23.505737 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 459/915] 10.222.0.55 () {56 vars in 1111 bytes} [Wed Aug 24 08:35:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:23.508105 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3f6334-b63e-4c0c-9ce0-b0cf66078370 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:23.524525 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 457/916] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 08:35:23 2022] POST /identity/v3/endpoints => generated 346 bytes in 367 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:35:23.725505 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3f6334-b63e-4c0c-9ce0-b0cf66078370 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:23.778079 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b65a2d-1e6c-495b-ae25-9b1a68b85bab None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:23.779473 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b65a2d-1e6c-495b-ae25-9b1a68b85bab None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:23.780265 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b65a2d-1e6c-495b-ae25-9b1a68b85bab None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:23.782984 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 458/917] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:35:23 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:35:23.789019 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9559d5-15a2-498d-9cc2-ab55687475d9 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:23.789510 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9559d5-15a2-498d-9cc2-ab55687475d9 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:23.789935 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9559d5-15a2-498d-9cc2-ab55687475d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:23.805501 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a12116c-9c17-4e9e-9e55-fe1ca68c4d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:23.807177 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a12116c-9c17-4e9e-9e55-fe1ca68c4d41 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:23.807612 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a12116c-9c17-4e9e-9e55-fe1ca68c4d41 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:23.808212 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a12116c-9c17-4e9e-9e55-fe1ca68c4d41 None admin] PATH_INFO: `/v3/projects` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:23.809641 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a12116c-9c17-4e9e-9e55-fe1ca68c4d41 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:23.810344 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a12116c-9c17-4e9e-9e55-fe1ca68c4d41 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:23.811629 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-ba9559d5-15a2-498d-9cc2-ab55687475d9 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:35:23.813077 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-ba9559d5-15a2-498d-9cc2-ab55687475d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:35:23.813614 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a12116c-9c17-4e9e-9e55-fe1ca68c4d41 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:23.843876 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 460/918] 10.222.0.55 () {56 vars in 1148 bytes} [Wed Aug 24 08:35:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:23.851190 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a12116c-9c17-4e9e-9e55-fe1ca68c4d41 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:23.908680 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 459/919] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:35:23 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 08:35:23.921080 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:24.059024 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a12116c-9c17-4e9e-9e55-fe1ca68c4d41 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:24.120662 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40853a19-615f-490d-a466-a5d46df8adfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:24.121888 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40853a19-615f-490d-a466-a5d46df8adfb None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:24.123034 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40853a19-615f-490d-a466-a5d46df8adfb None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:24.123643 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40853a19-615f-490d-a466-a5d46df8adfb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:24.125217 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40853a19-615f-490d-a466-a5d46df8adfb None admin] RBAC: Adding query filter params (role.id=371e1f8b73f144aea9bb7d20d253e5a0, scope.project.id=420b18725f2342159646fb86438318fd, user.id=f652a368b1cc42ae8b6f6c7a6dff6f2f) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:24.125935 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40853a19-615f-490d-a466-a5d46df8adfb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:24.129005 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40853a19-615f-490d-a466-a5d46df8adfb None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:24.141393 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 461/920] 10.222.0.55 () {56 vars in 1374 bytes} [Wed Aug 24 08:35:23 2022] GET /identity/v3/role_assignments?user.id=f652a368b1cc42ae8b6f6c7a6dff6f2f&scope.project.id=420b18725f2342159646fb86438318fd&role.id=371e1f8b73f144aea9bb7d20d253e5a0 => generated 259 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:24.181738 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9559d5-15a2-498d-9cc2-ab55687475d9 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:24.340908 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:24.389655 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9559d5-15a2-498d-9cc2-ab55687475d9 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:24.455662 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe8db6f-c6ee-48db-9985-3d369596c6a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:24.456777 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe8db6f-c6ee-48db-9985-3d369596c6a1 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:24.457416 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe8db6f-c6ee-48db-9985-3d369596c6a1 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:24.457898 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe8db6f-c6ee-48db-9985-3d369596c6a1 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:24.458964 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe8db6f-c6ee-48db-9985-3d369596c6a1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:24.461570 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe8db6f-c6ee-48db-9985-3d369596c6a1 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:24.476569 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-dbe8db6f-c6ee-48db-9985-3d369596c6a1 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 08:35:24.476569 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:24.476569 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 08:35:24.476569 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 08:35:24.476569 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 08:35:24.476569 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 08:35:24.476569 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 08:35:24.476569 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:35:24.476569 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 08:35:24.476569 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:35:24.476569 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:35:24.476569 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:35:24.476569 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:24.476569 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:24.476569 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:24.476569 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:24.477597 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:24.477597 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:24.477597 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:24.477597 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:24.477597 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:24.477597 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:24.477597 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:24.477597 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 08:35:24.477597 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 08:35:24.477597 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 08:35:24.477597 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 08:35:24.477597 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:24.477597 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:24.477597 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:24.477597 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:24.477597 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:24.477597 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:24.478495 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:24.478495 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:24.478495 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:24.478495 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:24.478495 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:24.478495 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:24.478495 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:24.478495 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:24.478495 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:24.478495 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:24.478495 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 08:35:24.478495 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:35:24.478495 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 08:35:24.478495 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:35:24.479509 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 460/921] 10.222.0.55 () {56 vars in 1132 bytes} [Wed Aug 24 08:35:24 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:24.486927 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40853a19-615f-490d-a466-a5d46df8adfb None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:24.605392 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:24.615843 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40853a19-615f-490d-a466-a5d46df8adfb None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:24.653808 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e6d9e5-35f2-4cf7-a9dd-19f7010db6a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:24.654431 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e6d9e5-35f2-4cf7-a9dd-19f7010db6a7 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:24.654708 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e6d9e5-35f2-4cf7-a9dd-19f7010db6a7 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:24.654981 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e6d9e5-35f2-4cf7-a9dd-19f7010db6a7 None admin] PATH_INFO: `/v3/services` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:24.655625 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e6d9e5-35f2-4cf7-a9dd-19f7010db6a7 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:24.655924 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e6d9e5-35f2-4cf7-a9dd-19f7010db6a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:24.657337 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e6d9e5-35f2-4cf7-a9dd-19f7010db6a7 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:24.663135 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 462/922] 10.222.0.55 () {56 vars in 1126 bytes} [Wed Aug 24 08:35:24 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:24.669839 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe8db6f-c6ee-48db-9985-3d369596c6a1 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:24.786920 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe8db6f-c6ee-48db-9985-3d369596c6a1 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:24.826667 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509fced9-e5a7-4e44-b508-31fafa10bb10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:24.827310 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509fced9-e5a7-4e44-b508-31fafa10bb10 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:24.827584 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509fced9-e5a7-4e44-b508-31fafa10bb10 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:24.827849 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509fced9-e5a7-4e44-b508-31fafa10bb10 None admin] PATH_INFO: `/v3/services` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:24.828564 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509fced9-e5a7-4e44-b508-31fafa10bb10 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:24.828875 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509fced9-e5a7-4e44-b508-31fafa10bb10 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:24.830292 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509fced9-e5a7-4e44-b508-31fafa10bb10 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:24.836270 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 461/923] 10.222.0.55 () {56 vars in 1126 bytes} [Wed Aug 24 08:35:24 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:24.843090 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e6d9e5-35f2-4cf7-a9dd-19f7010db6a7 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:24.965249 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e6d9e5-35f2-4cf7-a9dd-19f7010db6a7 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:25.008701 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4832ccf-c27d-4ff3-b299-40f51525528c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:25.009324 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4832ccf-c27d-4ff3-b299-40f51525528c None admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:25.009598 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4832ccf-c27d-4ff3-b299-40f51525528c None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:25.009873 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4832ccf-c27d-4ff3-b299-40f51525528c None admin] PATH_INFO: `/v3/endpoints` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:25.010615 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4832ccf-c27d-4ff3-b299-40f51525528c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:25.011983 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4832ccf-c27d-4ff3-b299-40f51525528c None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:25.101779 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 463/924] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 08:35:24 2022] POST /identity/v3/endpoints => generated 347 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:35:25.439485 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f67af0-e265-486e-a602-e9d9a3ecc85d None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:25.439876 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f67af0-e265-486e-a602-e9d9a3ecc85d None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:25.440055 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f67af0-e265-486e-a602-e9d9a3ecc85d None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:25.441659 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 462/925] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:35:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:35:25.447609 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d4586f-36ce-495f-8b2f-901475eaa13e None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:25.447887 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d4586f-36ce-495f-8b2f-901475eaa13e None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:25.448170 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d4586f-36ce-495f-8b2f-901475eaa13e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:25.471128 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-d9d4586f-36ce-495f-8b2f-901475eaa13e None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:35:25.473071 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-d9d4586f-36ce-495f-8b2f-901475eaa13e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:35:25.478208 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:25.605381 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 464/926] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:35:25 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:35:25.878428 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f67af0-e265-486e-a602-e9d9a3ecc85d None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:25.882062 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:25.940090 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2d0a80-ee19-4e1c-9a27-e0055e62a50c None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:25.940402 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2d0a80-ee19-4e1c-9a27-e0055e62a50c None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:25.940879 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2d0a80-ee19-4e1c-9a27-e0055e62a50c None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:25.942458 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 465/927] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:35:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:35:25.949520 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411384af-3eac-497b-a18b-fb0af8404bc2 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:25.949774 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411384af-3eac-497b-a18b-fb0af8404bc2 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:25.950041 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411384af-3eac-497b-a18b-fb0af8404bc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:25.973874 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-411384af-3eac-497b-a18b-fb0af8404bc2 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:35:25.975807 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-411384af-3eac-497b-a18b-fb0af8404bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:35:26.008354 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f67af0-e265-486e-a602-e9d9a3ecc85d None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:26.053251 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f3782e-b106-4d7a-b24f-802a7eff46ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:26.054434 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f3782e-b106-4d7a-b24f-802a7eff46ca None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:26.054717 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f3782e-b106-4d7a-b24f-802a7eff46ca None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:26.054992 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f3782e-b106-4d7a-b24f-802a7eff46ca None admin] PATH_INFO: `/v3/services/object-store` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:26.055595 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f3782e-b106-4d7a-b24f-802a7eff46ca None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:26.057332 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f3782e-b106-4d7a-b24f-802a7eff46ca None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:26.069781 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-17f3782e-b106-4d7a-b24f-802a7eff46ca None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 08:35:26.069781 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:26.069781 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 08:35:26.069781 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 08:35:26.069781 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 08:35:26.069781 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 08:35:26.069781 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 08:35:26.069781 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:35:26.069781 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 08:35:26.069781 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:35:26.069781 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:35:26.069781 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:35:26.069781 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:26.069781 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:26.069781 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:26.069781 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:26.069781 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:26.071519 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:26.071519 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:26.071519 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:26.071519 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:26.071519 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:26.071519 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:26.071519 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 08:35:26.071519 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 08:35:26.071519 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 08:35:26.071519 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 08:35:26.071519 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:26.071519 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:26.071519 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:26.071519 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:26.071519 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:26.071519 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:26.076957 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:26.076957 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:26.076957 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:26.076957 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:26.076957 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:26.076957 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:26.076957 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:26.076957 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:26.076957 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:26.076957 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:26.076957 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 08:35:26.076957 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:35:26.076957 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 08:35:26.076957 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:35:26.076957 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 463/928] 10.222.0.55 () {56 vars in 1126 bytes} [Wed Aug 24 08:35:25 2022] GET /identity/v3/services/object-store => generated 93 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:26.081836 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f3782e-b106-4d7a-b24f-802a7eff46ca None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:26.086251 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 466/929] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:35:25 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 08:35:26.089481 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d90abb9-b784-4dfa-a4eb-ac2b180b6d70 None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:26.089909 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d90abb9-b784-4dfa-a4eb-ac2b180b6d70 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:26.090353 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d90abb9-b784-4dfa-a4eb-ac2b180b6d70 None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:26.092041 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 467/930] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:35:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:35:26.100663 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a1c736-82ee-4639-adf6-d7a9f727e2d0 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:26.101384 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a1c736-82ee-4639-adf6-d7a9f727e2d0 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:26.102230 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a1c736-82ee-4639-adf6-d7a9f727e2d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:26.132008 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-26a1c736-82ee-4639-adf6-d7a9f727e2d0 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:35:26.133907 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-26a1c736-82ee-4639-adf6-d7a9f727e2d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:35:26.223354 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:26.245382 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:26.256832 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 468/931] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:35:26 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:35:26.277564 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f3782e-b106-4d7a-b24f-802a7eff46ca None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:26.325584 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d02706-1ef7-463a-9709-e8a362710530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:26.326448 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d02706-1ef7-463a-9709-e8a362710530 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:26.326733 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d02706-1ef7-463a-9709-e8a362710530 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:26.327160 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d02706-1ef7-463a-9709-e8a362710530 None admin] PATH_INFO: `/v3/services` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:26.328081 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d02706-1ef7-463a-9709-e8a362710530 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:26.328437 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d02706-1ef7-463a-9709-e8a362710530 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:26.329979 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d02706-1ef7-463a-9709-e8a362710530 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:26.337054 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 464/932] 10.222.0.55 () {56 vars in 1122 bytes} [Wed Aug 24 08:35:26 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:26.343384 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a1c736-82ee-4639-adf6-d7a9f727e2d0 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:26.376804 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d02706-1ef7-463a-9709-e8a362710530 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:26.470630 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a1c736-82ee-4639-adf6-d7a9f727e2d0 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:26.493826 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d02706-1ef7-463a-9709-e8a362710530 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:26.508869 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364efaa2-a53e-4834-8f4d-db1c8d840833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:26.511284 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364efaa2-a53e-4834-8f4d-db1c8d840833 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:26.511284 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364efaa2-a53e-4834-8f4d-db1c8d840833 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:26.511284 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364efaa2-a53e-4834-8f4d-db1c8d840833 None admin] PATH_INFO: `/v3/services` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:26.511284 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364efaa2-a53e-4834-8f4d-db1c8d840833 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:26.511284 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364efaa2-a53e-4834-8f4d-db1c8d840833 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:26.512592 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364efaa2-a53e-4834-8f4d-db1c8d840833 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:26.523014 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 469/933] 10.222.0.55 () {56 vars in 1122 bytes} [Wed Aug 24 08:35:26 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:26.524360 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364efaa2-a53e-4834-8f4d-db1c8d840833 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:26.532927 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:26.540972 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ae2c44-87e6-4da6-86f1-07915d438675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:26.541625 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ae2c44-87e6-4da6-86f1-07915d438675 None admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:26.541910 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ae2c44-87e6-4da6-86f1-07915d438675 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:26.542185 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ae2c44-87e6-4da6-86f1-07915d438675 None admin] PATH_INFO: `/v3/projects` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:26.542851 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ae2c44-87e6-4da6-86f1-07915d438675 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:26.545000 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ae2c44-87e6-4da6-86f1-07915d438675 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:26.547874 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: WARNING keystone.server.flask.common [None req-09ae2c44-87e6-4da6-86f1-07915d438675 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 24 08:35:26.606405 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:26.615310 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.sql.core [None req-09ae2c44-87e6-4da6-86f1-07915d438675 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 08:35:26.615310 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 24 08:35:26.615310 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [parameters: {'id': 'c76e08b88e6641ba9c99332b372fe8ae', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 24 08:35:26.615310 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=93542) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 08:35:26.623241 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-09ae2c44-87e6-4da6-86f1-07915d438675 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c76e08b88e6641ba9c99332b372fe8ae.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c76e08b88e6641ba9c99332b372fe8ae. Aug 24 08:35:26.623241 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:26.623241 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 08:35:26.623241 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 08:35:26.623241 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 08:35:26.623241 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 08:35:26.623241 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 08:35:26.623241 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 08:35:26.623241 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 08:35:26.623241 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application conn.query(q) Aug 24 08:35:26.623241 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 08:35:26.623241 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 08:35:26.623241 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 08:35:26.624901 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application result.read() Aug 24 08:35:26.624901 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 08:35:26.624901 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 08:35:26.624901 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 08:35:26.624901 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 08:35:26.624901 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 08:35:26.624901 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 08:35:26.624901 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 08:35:26.624901 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 08:35:26.624901 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 08:35:26.624901 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:35:26.624901 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 08:35:26.624901 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:35:26.624901 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:26.624901 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 08:35:26.624901 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 08:35:26.624901 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 24 08:35:26.624901 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 24 08:35:26.626893 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 08:35:26.626893 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application next(self.gen) Aug 24 08:35:26.626893 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 08:35:26.626893 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application yield resource Aug 24 08:35:26.626893 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 08:35:26.626893 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application next(self.gen) Aug 24 08:35:26.626893 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 08:35:26.626893 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 08:35:26.626893 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 08:35:26.626893 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 08:35:26.626893 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 08:35:26.626893 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise self.value Aug 24 08:35:26.626893 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 08:35:26.626893 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 08:35:26.626893 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 08:35:26.626893 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application session.commit() Aug 24 08:35:26.626893 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 08:35:26.629829 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 08:35:26.629829 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 08:35:26.629829 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 08:35:26.629829 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 08:35:26.629829 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application self.session.flush() Aug 24 08:35:26.629829 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 08:35:26.629829 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 08:35:26.629829 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 08:35:26.629829 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 08:35:26.629829 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 08:35:26.629829 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application compat.raise_( Aug 24 08:35:26.629829 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 08:35:26.629829 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception Aug 24 08:35:26.629829 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 08:35:26.629829 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 08:35:26.629829 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 08:35:26.629829 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 08:35:26.632509 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 08:35:26.632509 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 08:35:26.632509 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 08:35:26.632509 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 08:35:26.632509 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 24 08:35:26.632509 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 24 08:35:26.632509 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 08:35:26.632509 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 08:35:26.632509 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 08:35:26.632509 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 08:35:26.632509 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 08:35:26.632509 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 08:35:26.632509 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 08:35:26.632509 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 08:35:26.632509 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 08:35:26.633750 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 08:35:26.633750 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 08:35:26.633750 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception Aug 24 08:35:26.633750 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 08:35:26.633750 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 08:35:26.633750 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 08:35:26.633750 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 08:35:26.633750 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 08:35:26.633750 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 08:35:26.633750 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 08:35:26.633750 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application conn.query(q) Aug 24 08:35:26.633750 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 08:35:26.633750 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 08:35:26.633750 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 08:35:26.633750 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application result.read() Aug 24 08:35:26.633750 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 08:35:26.633750 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 08:35:26.642881 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 08:35:26.642881 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 08:35:26.642881 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 08:35:26.642881 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 08:35:26.642881 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 08:35:26.642881 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 08:35:26.642881 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 08:35:26.642881 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 24 08:35:26.642881 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [parameters: {'id': 'c76e08b88e6641ba9c99332b372fe8ae', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 24 08:35:26.642881 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 08:35:26.642881 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:35:26.642881 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:35:26.642881 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:35:26.642881 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:26.642881 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 24 08:35:26.644951 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 24 08:35:26.644951 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 08:35:26.644951 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 08:35:26.644951 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 24 08:35:26.644951 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:35:26.644951 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:35:26.644951 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:35:26.644951 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:26.644951 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:26.644951 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:26.644951 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:26.644951 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:26.644951 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:26.644951 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:26.644951 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:26.644951 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:26.644951 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:26.646850 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:26.646850 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 24 08:35:26.646850 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 24 08:35:26.646850 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:26.646850 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:26.646850 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 24 08:35:26.646850 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 24 08:35:26.646850 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c76e08b88e6641ba9c99332b372fe8ae. Aug 24 08:35:26.646850 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:35:26.646850 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 465/934] 10.222.0.55 () {60 vars in 1140 bytes} [Wed Aug 24 08:35:26 2022] POST /identity/v3/projects => generated 262 bytes in 248 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:26.646850 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ae2c44-87e6-4da6-86f1-07915d438675 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:26.672473 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364efaa2-a53e-4834-8f4d-db1c8d840833 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:26.710752 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b22df5-52b2-4edf-b26e-f698320ffdb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:26.711368 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b22df5-52b2-4edf-b26e-f698320ffdb9 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:26.711705 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b22df5-52b2-4edf-b26e-f698320ffdb9 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:26.712110 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b22df5-52b2-4edf-b26e-f698320ffdb9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:26.714019 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b22df5-52b2-4edf-b26e-f698320ffdb9 None admin] RBAC: Adding query filter params (interface=public, service_id=90a2f3f416014911aeb8a2aef87caf09, region_id=RegionOne) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:26.714019 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b22df5-52b2-4edf-b26e-f698320ffdb9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:26.715039 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b22df5-52b2-4edf-b26e-f698320ffdb9 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:26.725781 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 470/935] 10.222.0.55 () {56 vars in 1251 bytes} [Wed Aug 24 08:35:26 2022] GET /identity/v3/endpoints?service_id=90a2f3f416014911aeb8a2aef87caf09&interface=public®ion_id=RegionOne => generated 194 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:26.725781 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b22df5-52b2-4edf-b26e-f698320ffdb9 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:26.787141 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ae2c44-87e6-4da6-86f1-07915d438675 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:26.824721 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12158d0-858a-426a-9b8f-4c7c87167bce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:26.826361 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12158d0-858a-426a-9b8f-4c7c87167bce None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:26.826787 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12158d0-858a-426a-9b8f-4c7c87167bce None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:26.827200 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12158d0-858a-426a-9b8f-4c7c87167bce None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:26.839854 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b22df5-52b2-4edf-b26e-f698320ffdb9 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:26.840617 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12158d0-858a-426a-9b8f-4c7c87167bce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:35:26.847264 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-a12158d0-858a-426a-9b8f-4c7c87167bce None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 08:35:26.847264 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:26.847264 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:26.847264 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:26.847264 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:26.847264 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:26.847264 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:26.847264 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:26.847264 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:26.847264 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:26.847264 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:26.847264 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:26.847264 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:35:26.847264 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:35:26.847264 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:35:26.847264 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:35:26.848385 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:35:26.848385 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:35:26.848385 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:35:26.848385 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:35:26.848385 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:26.848385 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:26.848385 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:26.848385 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:26.848385 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:26.848385 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:26.848385 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:26.848385 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:26.848385 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:26.848385 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:26.848385 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:26.848385 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:26.849338 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:26.849338 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:26.849338 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:26.849338 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:26.849338 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:35:26.849338 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:35:26.849338 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:35:26.849338 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:35:26.849338 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:35:26.849338 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:35:26.849338 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 08:35:26.849338 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:35:26.853602 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 466/936] 10.222.0.55 () {56 vars in 1120 bytes} [Wed Aug 24 08:35:26 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:26.854342 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12158d0-858a-426a-9b8f-4c7c87167bce None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:26.875621 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:26.892638 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1967183f-44bd-4507-b491-e9ac06b6d06d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:26.893719 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1967183f-44bd-4507-b491-e9ac06b6d06d None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:26.894240 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1967183f-44bd-4507-b491-e9ac06b6d06d None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:26.894633 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1967183f-44bd-4507-b491-e9ac06b6d06d None admin] PATH_INFO: `/v3/projects/service` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:26.902447 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1967183f-44bd-4507-b491-e9ac06b6d06d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:26.904022 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1967183f-44bd-4507-b491-e9ac06b6d06d None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:26.912622 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:26.920838 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-1967183f-44bd-4507-b491-e9ac06b6d06d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:35:26.920838 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:26.920838 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:26.920838 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:26.920838 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:26.920838 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:26.920838 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:26.920838 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:26.920838 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:26.920838 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:26.920838 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:26.920838 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:26.920838 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:35:26.920838 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:35:26.920838 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:35:26.928159 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:35:26.928159 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:26.928159 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:26.928159 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:26.928159 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:26.928159 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:26.928159 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:26.928159 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:26.928159 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:26.928159 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:26.928159 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:26.928159 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:26.928159 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:26.928159 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:26.928159 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:26.928159 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:26.928159 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:26.929315 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:35:26.929315 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:35:26.929315 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:35:26.929315 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:35:26.929315 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:35:26.929315 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:35:26.929315 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:35:26.929315 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:35:26.929315 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 471/937] 10.222.0.55 () {56 vars in 1111 bytes} [Wed Aug 24 08:35:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:26.929315 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1967183f-44bd-4507-b491-e9ac06b6d06d None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:26.991263 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12158d0-858a-426a-9b8f-4c7c87167bce None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:27.031392 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884b02f9-78d7-4bca-bbf9-d2f23203aa01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:27.032087 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884b02f9-78d7-4bca-bbf9-d2f23203aa01 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:27.032384 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884b02f9-78d7-4bca-bbf9-d2f23203aa01 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:27.032749 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884b02f9-78d7-4bca-bbf9-d2f23203aa01 None admin] PATH_INFO: `/v3/services` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:27.033304 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884b02f9-78d7-4bca-bbf9-d2f23203aa01 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:27.034908 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884b02f9-78d7-4bca-bbf9-d2f23203aa01 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:27.058824 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 467/938] 10.222.0.55 () {56 vars in 1087 bytes} [Wed Aug 24 08:35:26 2022] GET /identity/v3/services => generated 1497 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:35:27.058824 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884b02f9-78d7-4bca-bbf9-d2f23203aa01 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:27.077923 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1967183f-44bd-4507-b491-e9ac06b6d06d None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:27.117379 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6c0c12-5b98-476c-b7b1-1437c6ce350c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:27.118189 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6c0c12-5b98-476c-b7b1-1437c6ce350c None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:27.118526 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6c0c12-5b98-476c-b7b1-1437c6ce350c None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:27.118873 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6c0c12-5b98-476c-b7b1-1437c6ce350c None admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:27.119649 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6c0c12-5b98-476c-b7b1-1437c6ce350c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:27.120034 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6c0c12-5b98-476c-b7b1-1437c6ce350c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:27.121648 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6c0c12-5b98-476c-b7b1-1437c6ce350c None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:27.137057 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 472/939] 10.222.0.55 () {56 vars in 1115 bytes} [Wed Aug 24 08:35:26 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:27.139838 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6c0c12-5b98-476c-b7b1-1437c6ce350c None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:27.195941 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:27.199791 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884b02f9-78d7-4bca-bbf9-d2f23203aa01 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:27.217159 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:27.255366 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319bb794-c57b-41f8-b84c-d547ce433513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:27.257069 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319bb794-c57b-41f8-b84c-d547ce433513 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:27.257645 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319bb794-c57b-41f8-b84c-d547ce433513 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:27.258229 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319bb794-c57b-41f8-b84c-d547ce433513 None admin] PATH_INFO: `/v3/projects` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:27.259686 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319bb794-c57b-41f8-b84c-d547ce433513 None admin] RBAC: Adding query filter params (name=service) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:27.260267 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319bb794-c57b-41f8-b84c-d547ce433513 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:27.262926 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6c0c12-5b98-476c-b7b1-1437c6ce350c None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:27.264529 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319bb794-c57b-41f8-b84c-d547ce433513 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:27.289198 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 468/940] 10.222.0.55 () {56 vars in 1112 bytes} [Wed Aug 24 08:35:27 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:27.290501 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7917a3b9-e45b-4357-837d-ea84de2cd2e6 None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:27.290790 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7917a3b9-e45b-4357-837d-ea84de2cd2e6 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:27.291045 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7917a3b9-e45b-4357-837d-ea84de2cd2e6 None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:27.292562 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 469/941] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:35:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:35:27.298825 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1d935c-002a-4308-aa75-7d025d1f4a65 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:27.299221 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1d935c-002a-4308-aa75-7d025d1f4a65 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:27.299496 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1d935c-002a-4308-aa75-7d025d1f4a65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:27.316801 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da87d643-d323-402c-8caa-95d389f1571f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:27.317491 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da87d643-d323-402c-8caa-95d389f1571f None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:27.317778 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da87d643-d323-402c-8caa-95d389f1571f None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:27.318134 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da87d643-d323-402c-8caa-95d389f1571f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:27.320072 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-5a1d935c-002a-4308-aa75-7d025d1f4a65 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:35:27.322556 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-5a1d935c-002a-4308-aa75-7d025d1f4a65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:35:27.339236 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da87d643-d323-402c-8caa-95d389f1571f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:27.342139 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da87d643-d323-402c-8caa-95d389f1571f None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:27.375868 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-da87d643-d323-402c-8caa-95d389f1571f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:27.375868 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:27.375868 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:27.375868 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:27.375868 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:27.375868 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:27.375868 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:27.375868 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:27.375868 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:27.375868 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:27.375868 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:27.375868 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:27.375868 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:35:27.375868 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:35:27.375868 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:35:27.375868 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:35:27.376961 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:27.376961 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:27.376961 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:27.376961 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:27.376961 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:27.376961 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:27.376961 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:27.376961 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:27.376961 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:27.376961 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:27.376961 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:27.376961 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:27.376961 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:27.376961 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:27.376961 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:27.376961 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:27.376961 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:35:27.377860 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:35:27.377860 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:27.377860 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:35:27.378048 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 473/942] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:35:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:35:27.385711 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da87d643-d323-402c-8caa-95d389f1571f None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:27.425238 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 470/943] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:35:27 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:35:27.582760 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:27.586843 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da87d643-d323-402c-8caa-95d389f1571f None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:27.632039 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f59194-84fa-4c2e-9a53-7d77dd6bd966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:27.633009 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f59194-84fa-4c2e-9a53-7d77dd6bd966 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:27.633359 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f59194-84fa-4c2e-9a53-7d77dd6bd966 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:27.633640 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f59194-84fa-4c2e-9a53-7d77dd6bd966 None admin] PATH_INFO: `/v3/domains` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:27.634412 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f59194-84fa-4c2e-9a53-7d77dd6bd966 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:27.634811 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f59194-84fa-4c2e-9a53-7d77dd6bd966 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:27.636224 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f59194-84fa-4c2e-9a53-7d77dd6bd966 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:27.659286 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 474/944] 10.222.0.55 () {56 vars in 1109 bytes} [Wed Aug 24 08:35:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:27.664866 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1d935c-002a-4308-aa75-7d025d1f4a65 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:27.708779 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f59194-84fa-4c2e-9a53-7d77dd6bd966 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:27.796987 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:27.800145 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1d935c-002a-4308-aa75-7d025d1f4a65 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:27.846509 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9c7317-b9c2-446d-98f2-4c45bbb846c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:27.847601 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9c7317-b9c2-446d-98f2-4c45bbb846c5 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:27.847885 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9c7317-b9c2-446d-98f2-4c45bbb846c5 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:27.848437 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9c7317-b9c2-446d-98f2-4c45bbb846c5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:27.859424 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9c7317-b9c2-446d-98f2-4c45bbb846c5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:27.859692 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f59194-84fa-4c2e-9a53-7d77dd6bd966 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:27.860907 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9c7317-b9c2-446d-98f2-4c45bbb846c5 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:27.878622 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-3b9c7317-b9c2-446d-98f2-4c45bbb846c5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 08:35:27.878622 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:27.878622 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:27.878622 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:27.878622 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:27.878622 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:27.878622 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:27.878622 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:27.878622 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:27.878622 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:27.878622 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:27.878622 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:27.878622 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:35:27.878622 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:35:27.878622 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:35:27.878622 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:35:27.879660 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:27.879660 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:27.879660 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:35:27.879660 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:35:27.879660 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:35:27.879660 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:35:27.879660 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:27.879660 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:27.879660 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:27.879660 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:27.879660 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:27.879660 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:27.879660 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:27.879660 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:27.879660 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:27.879660 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:27.879660 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:27.879660 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:27.880653 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:27.880653 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:27.880653 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:35:27.880653 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:35:27.880653 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:35:27.880653 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:35:27.880653 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:35:27.880653 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:35:27.880653 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 08:35:27.880653 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:35:27.885032 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 471/945] 10.222.0.55 () {56 vars in 1117 bytes} [Wed Aug 24 08:35:27 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:27.887287 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9c7317-b9c2-446d-98f2-4c45bbb846c5 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:27.914011 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e2ce83-6f0c-4bfe-b4d2-15c6414443d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:27.914652 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e2ce83-6f0c-4bfe-b4d2-15c6414443d6 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:27.914993 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e2ce83-6f0c-4bfe-b4d2-15c6414443d6 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:27.915291 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e2ce83-6f0c-4bfe-b4d2-15c6414443d6 None admin] PATH_INFO: `/v3/services/compute` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:27.915911 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e2ce83-6f0c-4bfe-b4d2-15c6414443d6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:27.917960 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e2ce83-6f0c-4bfe-b4d2-15c6414443d6 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:27.931006 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-d7e2ce83-6f0c-4bfe-b4d2-15c6414443d6 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 08:35:27.931006 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:27.931006 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 08:35:27.931006 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 08:35:27.931006 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 08:35:27.931006 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 08:35:27.931006 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 08:35:27.931006 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:35:27.931006 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 08:35:27.931006 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:35:27.931006 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:35:27.931006 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:35:27.931006 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:27.931006 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:27.931006 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:27.931006 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:27.931006 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:27.932099 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:27.932099 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:27.932099 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:27.932099 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:27.932099 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:27.932099 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:27.932099 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 08:35:27.932099 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 08:35:27.932099 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 08:35:27.932099 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 08:35:27.932099 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:27.932099 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:27.932099 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:27.932099 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:27.932099 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:27.932099 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:27.933011 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:27.933011 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:27.933011 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:27.933011 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:27.933011 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:27.933011 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:27.933011 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:27.933011 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:27.933011 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:27.933011 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:27.933011 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 08:35:27.933011 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:35:27.933011 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 08:35:27.933011 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:35:27.938111 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 475/946] 10.222.0.55 () {56 vars in 1111 bytes} [Wed Aug 24 08:35:27 2022] GET /identity/v3/services/compute => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:35:27.940116 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e2ce83-6f0c-4bfe-b4d2-15c6414443d6 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:27.980286 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:28.031185 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9c7317-b9c2-446d-98f2-4c45bbb846c5 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:28.078625 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14296e98-3884-4113-a989-193805fdd238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:28.079637 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14296e98-3884-4113-a989-193805fdd238 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:28.080124 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14296e98-3884-4113-a989-193805fdd238 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:28.080635 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14296e98-3884-4113-a989-193805fdd238 None admin] PATH_INFO: `/v3/users` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:28.081610 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14296e98-3884-4113-a989-193805fdd238 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:28.082634 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14296e98-3884-4113-a989-193805fdd238 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:28.084752 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14296e98-3884-4113-a989-193805fdd238 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:28.110615 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 472/947] 10.222.0.55 () {56 vars in 1149 bytes} [Wed Aug 24 08:35:27 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:28.111920 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14296e98-3884-4113-a989-193805fdd238 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:28.125657 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e2ce83-6f0c-4bfe-b4d2-15c6414443d6 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:28.157353 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:28.199675 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b5f922-8084-4ccc-9a4d-16c0cda84ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:28.201654 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b5f922-8084-4ccc-9a4d-16c0cda84ebb None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:28.201967 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b5f922-8084-4ccc-9a4d-16c0cda84ebb None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:28.202292 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b5f922-8084-4ccc-9a4d-16c0cda84ebb None admin] PATH_INFO: `/v3/services` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:28.203231 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b5f922-8084-4ccc-9a4d-16c0cda84ebb None admin] RBAC: Adding query filter params (name=compute) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:28.203638 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b5f922-8084-4ccc-9a4d-16c0cda84ebb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:28.208162 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b5f922-8084-4ccc-9a4d-16c0cda84ebb None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:28.233308 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 476/948] 10.222.0.55 () {56 vars in 1112 bytes} [Wed Aug 24 08:35:27 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:28.240736 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b5f922-8084-4ccc-9a4d-16c0cda84ebb None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:28.241605 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14296e98-3884-4113-a989-193805fdd238 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:28.281629 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874a1b34-01b8-425f-9478-b95d4011db73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:28.282654 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874a1b34-01b8-425f-9478-b95d4011db73 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:28.283171 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874a1b34-01b8-425f-9478-b95d4011db73 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:28.283669 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874a1b34-01b8-425f-9478-b95d4011db73 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:28.295548 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874a1b34-01b8-425f-9478-b95d4011db73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:28.297415 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874a1b34-01b8-425f-9478-b95d4011db73 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:28.313460 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-874a1b34-01b8-425f-9478-b95d4011db73 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:28.313460 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:28.313460 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:28.313460 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:28.313460 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:28.313460 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:28.313460 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:28.313460 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:28.313460 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:28.313460 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:28.313460 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:28.313460 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:28.313460 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:35:28.313460 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:35:28.313460 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:35:28.313460 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:35:28.314512 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:28.314512 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:28.314512 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:28.314512 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:28.314512 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:28.314512 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:28.314512 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:28.314512 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:28.314512 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:28.314512 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:28.314512 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:28.314512 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:28.314512 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:28.314512 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:28.314512 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:28.314512 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:28.314512 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:35:28.315462 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:35:28.315462 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:28.315462 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:35:28.322511 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 473/949] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:35:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:28.323800 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874a1b34-01b8-425f-9478-b95d4011db73 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:28.389427 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:28.405539 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:28.418873 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b5f922-8084-4ccc-9a4d-16c0cda84ebb None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:28.441029 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874a1b34-01b8-425f-9478-b95d4011db73 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:28.479367 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4adb89-bb21-4857-b6c4-6698b57738f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:28.480116 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa1bf30-aaaa-40d0-beb9-73cf9fe68767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:28.480752 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4adb89-bb21-4857-b6c4-6698b57738f2 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:28.481284 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa1bf30-aaaa-40d0-beb9-73cf9fe68767 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:28.481700 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4adb89-bb21-4857-b6c4-6698b57738f2 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:28.482216 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa1bf30-aaaa-40d0-beb9-73cf9fe68767 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:28.482474 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4adb89-bb21-4857-b6c4-6698b57738f2 None admin] PATH_INFO: `/v3/domains` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:28.483074 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa1bf30-aaaa-40d0-beb9-73cf9fe68767 None admin] PATH_INFO: `/v3/services` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:28.483989 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4adb89-bb21-4857-b6c4-6698b57738f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:28.484182 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa1bf30-aaaa-40d0-beb9-73cf9fe68767 None admin] RBAC: Adding query filter params (type=compute) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:28.484491 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa1bf30-aaaa-40d0-beb9-73cf9fe68767 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:28.484940 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4adb89-bb21-4857-b6c4-6698b57738f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:28.486270 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa1bf30-aaaa-40d0-beb9-73cf9fe68767 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:28.486830 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4adb89-bb21-4857-b6c4-6698b57738f2 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:28.500741 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 477/950] 10.222.0.55 () {56 vars in 1112 bytes} [Wed Aug 24 08:35:28 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:28.507730 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 474/951] 10.222.0.55 () {56 vars in 1109 bytes} [Wed Aug 24 08:35:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:28.509241 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa1bf30-aaaa-40d0-beb9-73cf9fe68767 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:28.700189 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa1bf30-aaaa-40d0-beb9-73cf9fe68767 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:28.768494 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9695add0-e784-4d46-9c7d-3fc2ad1d7f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:28.769621 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9695add0-e784-4d46-9c7d-3fc2ad1d7f81 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:28.770117 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9695add0-e784-4d46-9c7d-3fc2ad1d7f81 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:28.770605 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9695add0-e784-4d46-9c7d-3fc2ad1d7f81 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:28.783530 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9695add0-e784-4d46-9c7d-3fc2ad1d7f81 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:28.786434 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9695add0-e784-4d46-9c7d-3fc2ad1d7f81 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:28.808720 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-9695add0-e784-4d46-9c7d-3fc2ad1d7f81 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:35:28.808720 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:28.808720 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:28.808720 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:28.808720 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:28.808720 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:28.808720 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:28.808720 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:28.808720 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:28.808720 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:28.808720 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:28.808720 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:28.808720 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:35:28.808720 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:35:28.808720 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:35:28.809760 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:35:28.809760 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:28.809760 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:28.809760 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:28.809760 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:28.809760 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:28.809760 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:28.809760 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:28.809760 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:28.809760 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:28.809760 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:28.809760 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:28.809760 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:28.809760 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:28.809760 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:28.809760 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:28.809760 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:28.811286 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:35:28.811286 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:35:28.811286 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:35:28.811286 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:35:28.811286 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:35:28.811286 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:35:28.811286 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:35:28.811286 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:35:28.811814 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 478/952] 10.222.0.55 () {56 vars in 1111 bytes} [Wed Aug 24 08:35:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:28.818377 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4adb89-bb21-4857-b6c4-6698b57738f2 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:28.936357 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4adb89-bb21-4857-b6c4-6698b57738f2 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:28.972813 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:28.976641 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3d75be-fb13-4986-a1fb-df6b21839bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:28.977587 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3d75be-fb13-4986-a1fb-df6b21839bb3 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:28.978074 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3d75be-fb13-4986-a1fb-df6b21839bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:28.978537 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3d75be-fb13-4986-a1fb-df6b21839bb3 None admin] PATH_INFO: `/v3/projects` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:28.979438 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3d75be-fb13-4986-a1fb-df6b21839bb3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:28.979945 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3d75be-fb13-4986-a1fb-df6b21839bb3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:28.982299 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3d75be-fb13-4986-a1fb-df6b21839bb3 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:28.996095 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 475/953] 10.222.0.55 () {56 vars in 1148 bytes} [Wed Aug 24 08:35:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:29.003850 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9695add0-e784-4d46-9c7d-3fc2ad1d7f81 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:29.021271 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:29.126334 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9695add0-e784-4d46-9c7d-3fc2ad1d7f81 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:29.164678 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede5d380-dfae-4d41-8cd3-1bbe10604768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:29.165610 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede5d380-dfae-4d41-8cd3-1bbe10604768 None admin] REQUEST_METHOD: `PUT` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:29.166069 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede5d380-dfae-4d41-8cd3-1bbe10604768 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:29.166494 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede5d380-dfae-4d41-8cd3-1bbe10604768 None admin] PATH_INFO: `/v3/projects/420b18725f2342159646fb86438318fd/users/f652a368b1cc42ae8b6f6c7a6dff6f2f/roles/371e1f8b73f144aea9bb7d20d253e5a0` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:29.168959 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede5d380-dfae-4d41-8cd3-1bbe10604768 None admin] RBAC: Authorizing `identity:create_grant(project_id=420b18725f2342159646fb86438318fd, user_id=f652a368b1cc42ae8b6f6c7a6dff6f2f, role_id=371e1f8b73f144aea9bb7d20d253e5a0)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:29.171198 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede5d380-dfae-4d41-8cd3-1bbe10604768 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:29.278569 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 479/954] 10.222.0.55 () {58 vars in 1439 bytes} [Wed Aug 24 08:35:29 2022] PUT /identity/v3/projects/420b18725f2342159646fb86438318fd/users/f652a368b1cc42ae8b6f6c7a6dff6f2f/roles/371e1f8b73f144aea9bb7d20d253e5a0 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:35:29.468820 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab3a7a8-b32a-44a0-97ed-2d799ca4977c None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:29.469604 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab3a7a8-b32a-44a0-97ed-2d799ca4977c None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:29.469604 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab3a7a8-b32a-44a0-97ed-2d799ca4977c None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:29.471035 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 476/955] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:35:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:35:29.476993 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa74d591-f0ae-45d0-8880-d2166a19aef3 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:29.477325 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa74d591-f0ae-45d0-8880-d2166a19aef3 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:29.477520 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa74d591-f0ae-45d0-8880-d2166a19aef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:29.483842 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cbc3b1-0cf4-4b10-ba21-066153cd51df None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:29.484107 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cbc3b1-0cf4-4b10-ba21-066153cd51df None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:29.484407 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cbc3b1-0cf4-4b10-ba21-066153cd51df None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:29.486037 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 477/956] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:35:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:35:29.494590 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dc285c-0bdf-4098-b2f8-54fb88c94b56 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:29.494973 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dc285c-0bdf-4098-b2f8-54fb88c94b56 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:29.498194 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dc285c-0bdf-4098-b2f8-54fb88c94b56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:29.498645 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-fa74d591-f0ae-45d0-8880-d2166a19aef3 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:35:29.499884 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-fa74d591-f0ae-45d0-8880-d2166a19aef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:35:29.519557 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-89dc285c-0bdf-4098-b2f8-54fb88c94b56 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:35:29.521211 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-89dc285c-0bdf-4098-b2f8-54fb88c94b56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:35:29.574928 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:29.583507 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 480/957] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:35:29 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:35:29.616132 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 478/958] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:35:29 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 08:35:29.877530 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa74d591-f0ae-45d0-8880-d2166a19aef3 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:29.932388 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dc285c-0bdf-4098-b2f8-54fb88c94b56 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:30.081489 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dc285c-0bdf-4098-b2f8-54fb88c94b56 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:30.101564 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa74d591-f0ae-45d0-8880-d2166a19aef3 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:30.111243 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:30.127975 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caa1b0b-9b0a-427c-b934-49086ae9fc1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:30.129010 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caa1b0b-9b0a-427c-b934-49086ae9fc1b None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:30.129491 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caa1b0b-9b0a-427c-b934-49086ae9fc1b None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:30.130044 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caa1b0b-9b0a-427c-b934-49086ae9fc1b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:30.130855 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caa1b0b-9b0a-427c-b934-49086ae9fc1b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:30.132807 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caa1b0b-9b0a-427c-b934-49086ae9fc1b None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:30.143691 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-9caa1b0b-9b0a-427c-b934-49086ae9fc1b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 08:35:30.143691 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:30.143691 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 08:35:30.143691 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 08:35:30.143691 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 08:35:30.143691 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 08:35:30.143691 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 08:35:30.143691 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:35:30.143691 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 08:35:30.143691 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:35:30.143691 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:35:30.143691 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:35:30.143691 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:30.143691 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:30.143691 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:30.143691 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:30.143691 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:30.144889 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:30.144889 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:30.144889 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:30.144889 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:30.144889 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:30.144889 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:30.144889 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 08:35:30.144889 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 08:35:30.144889 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 08:35:30.144889 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 08:35:30.144889 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:30.144889 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:30.144889 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:30.144889 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:30.144889 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:30.144889 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:30.145756 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:30.145756 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:30.145756 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:30.145756 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:30.145756 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:30.145756 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:30.145756 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:30.145756 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:30.145756 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:30.145756 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:30.145756 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 08:35:30.145756 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:35:30.145756 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 08:35:30.145756 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:35:30.152951 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 479/959] 10.222.0.55 () {56 vars in 1126 bytes} [Wed Aug 24 08:35:29 2022] GET /identity/v3/services/object-store => generated 93 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:30.155041 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caa1b0b-9b0a-427c-b934-49086ae9fc1b None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:30.183414 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b43c2a-ada9-4335-9341-beb4c7ae3157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:30.184447 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b43c2a-ada9-4335-9341-beb4c7ae3157 None admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:30.188508 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b43c2a-ada9-4335-9341-beb4c7ae3157 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:30.189044 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b43c2a-ada9-4335-9341-beb4c7ae3157 None admin] PATH_INFO: `/v3/users` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:30.191260 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b43c2a-ada9-4335-9341-beb4c7ae3157 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:30.195275 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b43c2a-ada9-4335-9341-beb4c7ae3157 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:30.201803 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: WARNING keystone.server.flask.common [None req-d3b43c2a-ada9-4335-9341-beb4c7ae3157 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 24 08:35:30.260902 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.sql.core [None req-d3b43c2a-ada9-4335-9341-beb4c7ae3157 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 08:35:30.260902 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 24 08:35:30.260902 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [parameters: {'user_id': 'f34f15f87cf541a0a8d816b1dfc8d2f1', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 24 08:35:30.260902 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=93543) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 08:35:30.289739 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caa1b0b-9b0a-427c-b934-49086ae9fc1b None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:30.309428 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-d3b43c2a-ada9-4335-9341-beb4c7ae3157 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 24 08:35:30.309428 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:30.309428 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 08:35:30.309428 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 08:35:30.309428 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 08:35:30.309428 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 08:35:30.309428 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 08:35:30.309428 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 08:35:30.309428 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 08:35:30.309428 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application conn.query(q) Aug 24 08:35:30.309428 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 08:35:30.309428 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 08:35:30.309428 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 08:35:30.309428 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application result.read() Aug 24 08:35:30.309428 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 08:35:30.312330 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 08:35:30.312330 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 08:35:30.312330 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 08:35:30.312330 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 08:35:30.312330 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 08:35:30.312330 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 08:35:30.312330 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 08:35:30.312330 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 08:35:30.312330 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:35:30.312330 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 08:35:30.312330 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:35:30.312330 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:30.312330 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 08:35:30.312330 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 08:35:30.312330 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 24 08:35:30.312330 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 24 08:35:30.312330 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 08:35:30.312330 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application next(self.gen) Aug 24 08:35:30.313399 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 08:35:30.313399 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application yield resource Aug 24 08:35:30.313399 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 08:35:30.313399 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application next(self.gen) Aug 24 08:35:30.313399 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 08:35:30.313399 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 08:35:30.313399 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 08:35:30.313399 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 08:35:30.313399 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 08:35:30.313399 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise self.value Aug 24 08:35:30.313399 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 08:35:30.313399 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 08:35:30.313399 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 08:35:30.313399 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application session.commit() Aug 24 08:35:30.313399 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 08:35:30.313399 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 08:35:30.313399 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 08:35:30.314345 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 08:35:30.314345 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 08:35:30.314345 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application self.session.flush() Aug 24 08:35:30.314345 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 08:35:30.314345 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 08:35:30.314345 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 08:35:30.314345 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 08:35:30.314345 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 08:35:30.314345 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application compat.raise_( Aug 24 08:35:30.314345 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 08:35:30.314345 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception Aug 24 08:35:30.314345 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 08:35:30.314345 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 08:35:30.314345 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 08:35:30.314345 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 08:35:30.314345 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 08:35:30.314345 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 08:35:30.315244 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 08:35:30.315244 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 08:35:30.315244 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 24 08:35:30.315244 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 24 08:35:30.315244 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 08:35:30.315244 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 08:35:30.315244 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 08:35:30.315244 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 08:35:30.315244 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 08:35:30.315244 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 08:35:30.315244 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 08:35:30.315244 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 08:35:30.315244 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 08:35:30.315244 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 08:35:30.315244 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 08:35:30.315244 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception Aug 24 08:35:30.317658 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 08:35:30.317658 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 08:35:30.317658 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 08:35:30.317658 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 08:35:30.317658 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 08:35:30.317658 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 08:35:30.317658 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 08:35:30.317658 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application conn.query(q) Aug 24 08:35:30.317658 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 08:35:30.317658 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 08:35:30.317658 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 08:35:30.317658 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application result.read() Aug 24 08:35:30.317658 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 08:35:30.317658 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 08:35:30.317658 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 08:35:30.317658 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 08:35:30.319722 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 08:35:30.319722 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 08:35:30.319722 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 08:35:30.319722 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 08:35:30.319722 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 08:35:30.319722 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 24 08:35:30.319722 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [parameters: {'user_id': 'f34f15f87cf541a0a8d816b1dfc8d2f1', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 24 08:35:30.319722 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 08:35:30.319722 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:35:30.319722 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:35:30.319722 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:35:30.319722 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:30.319722 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:30.319722 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:30.319722 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:30.322079 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:30.322079 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:30.322079 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:30.322079 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:30.322079 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:30.322079 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:30.322079 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:30.322079 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 24 08:35:30.322079 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 24 08:35:30.322079 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:30.322079 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:30.322079 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:35:30.322079 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:35:30.322079 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:35:30.322079 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:35:30.322079 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 24 08:35:30.322079 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 24 08:35:30.323223 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 24 08:35:30.323223 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 24 08:35:30.323223 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 08:35:30.323223 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 08:35:30.323223 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 24 08:35:30.323223 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:35:30.323223 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 481/960] 10.222.0.55 () {60 vars in 1131 bytes} [Wed Aug 24 08:35:29 2022] POST /identity/v3/users => generated 160 bytes in 442 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:30.328521 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b43c2a-ada9-4335-9341-beb4c7ae3157 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:30.351532 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc63c8b-6ac6-4e73-bb2e-64e2a2ebe56d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:30.352466 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc63c8b-6ac6-4e73-bb2e-64e2a2ebe56d None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:30.352949 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc63c8b-6ac6-4e73-bb2e-64e2a2ebe56d None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:30.353424 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc63c8b-6ac6-4e73-bb2e-64e2a2ebe56d None admin] PATH_INFO: `/v3/services` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:30.354251 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc63c8b-6ac6-4e73-bb2e-64e2a2ebe56d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:30.355387 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc63c8b-6ac6-4e73-bb2e-64e2a2ebe56d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:30.357618 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc63c8b-6ac6-4e73-bb2e-64e2a2ebe56d None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:30.369650 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 480/961] 10.222.0.55 () {56 vars in 1122 bytes} [Wed Aug 24 08:35:30 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:35:30.370947 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc63c8b-6ac6-4e73-bb2e-64e2a2ebe56d None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:30.441035 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:30.499753 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc63c8b-6ac6-4e73-bb2e-64e2a2ebe56d None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:30.501665 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:30.542387 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1282a3a0-57e5-4337-92c5-4ac69c65e2c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:30.543354 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1282a3a0-57e5-4337-92c5-4ac69c65e2c5 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:30.543810 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1282a3a0-57e5-4337-92c5-4ac69c65e2c5 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:30.544265 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1282a3a0-57e5-4337-92c5-4ac69c65e2c5 None admin] PATH_INFO: `/v3/services` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:30.545688 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1282a3a0-57e5-4337-92c5-4ac69c65e2c5 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:30.545990 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1282a3a0-57e5-4337-92c5-4ac69c65e2c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:30.547377 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1282a3a0-57e5-4337-92c5-4ac69c65e2c5 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:30.558755 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 481/962] 10.222.0.55 () {56 vars in 1122 bytes} [Wed Aug 24 08:35:30 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:30.559920 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1282a3a0-57e5-4337-92c5-4ac69c65e2c5 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:30.610162 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b43c2a-ada9-4335-9341-beb4c7ae3157 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:30.698450 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1282a3a0-57e5-4337-92c5-4ac69c65e2c5 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:30.702002 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399f7a1e-34e0-4bdc-8d17-9e3b33c1a652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:30.703079 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399f7a1e-34e0-4bdc-8d17-9e3b33c1a652 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:30.703528 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399f7a1e-34e0-4bdc-8d17-9e3b33c1a652 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:30.704007 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399f7a1e-34e0-4bdc-8d17-9e3b33c1a652 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:30.719016 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399f7a1e-34e0-4bdc-8d17-9e3b33c1a652 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:30.722154 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399f7a1e-34e0-4bdc-8d17-9e3b33c1a652 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:30.724562 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:30.739748 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-399f7a1e-34e0-4bdc-8d17-9e3b33c1a652 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 08:35:30.739748 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:30.739748 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:30.739748 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:30.739748 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:30.739748 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:30.739748 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:30.739748 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:30.739748 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:30.739748 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:30.739748 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:30.739748 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:30.739748 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:35:30.739748 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:35:30.739748 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:35:30.739748 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:35:30.750942 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:30.750942 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:30.750942 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:35:30.750942 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:35:30.750942 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:35:30.750942 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:35:30.750942 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:30.750942 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:30.750942 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:30.750942 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:30.750942 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:30.750942 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:30.750942 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:30.750942 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:30.750942 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:30.750942 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:30.750942 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:30.750942 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:30.753434 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:30.753434 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:30.753434 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:35:30.753434 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:35:30.753434 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:35:30.753434 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:35:30.753434 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:35:30.753434 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:35:30.753434 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 08:35:30.753434 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:35:30.753434 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 482/963] 10.222.0.55 () {56 vars in 1099 bytes} [Wed Aug 24 08:35:30 2022] GET /identity/v3/users/cinder => generated 84 bytes in 415 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:30.753434 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a1b98e-f27e-45f0-83e1-8331300d6c6e None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:30.753434 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a1b98e-f27e-45f0-83e1-8331300d6c6e None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:30.754969 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a1b98e-f27e-45f0-83e1-8331300d6c6e None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:30.754969 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 483/964] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:35:30 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:35:30.754969 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eb890f-e17a-420d-baa2-d38c1faa1331 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:30.754969 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eb890f-e17a-420d-baa2-d38c1faa1331 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:30.754969 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eb890f-e17a-420d-baa2-d38c1faa1331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:30.776060 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-b9eb890f-e17a-420d-baa2-d38c1faa1331 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:35:30.777700 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-b9eb890f-e17a-420d-baa2-d38c1faa1331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:35:30.779917 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d41735-8c23-4246-b3c0-c7d787eea07d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:30.782711 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d41735-8c23-4246-b3c0-c7d787eea07d None admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:30.783279 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d41735-8c23-4246-b3c0-c7d787eea07d None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:30.783712 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d41735-8c23-4246-b3c0-c7d787eea07d None admin] PATH_INFO: `/v3/endpoints` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:30.785041 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d41735-8c23-4246-b3c0-c7d787eea07d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:30.788529 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d41735-8c23-4246-b3c0-c7d787eea07d None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:30.857817 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 484/965] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:35:30 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 08:35:30.858686 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eb890f-e17a-420d-baa2-d38c1faa1331 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:30.896682 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 482/966] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 08:35:30 2022] POST /identity/v3/endpoints => generated 349 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:35:30.954436 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:31.038209 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eb890f-e17a-420d-baa2-d38c1faa1331 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:31.116621 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fa4138-b49a-47ad-93a3-53b57dc7dd2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:31.117740 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fa4138-b49a-47ad-93a3-53b57dc7dd2d None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:31.118214 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fa4138-b49a-47ad-93a3-53b57dc7dd2d None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:31.118760 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fa4138-b49a-47ad-93a3-53b57dc7dd2d None admin] PATH_INFO: `/v3/users` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:31.120121 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fa4138-b49a-47ad-93a3-53b57dc7dd2d None admin] RBAC: Adding query filter params (name=cinder) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:31.120659 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fa4138-b49a-47ad-93a3-53b57dc7dd2d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:31.123480 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fa4138-b49a-47ad-93a3-53b57dc7dd2d None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:31.191205 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 485/967] 10.222.0.55 () {56 vars in 1101 bytes} [Wed Aug 24 08:35:30 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:31.349034 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d41735-8c23-4246-b3c0-c7d787eea07d None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:31.432361 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314ee23d-c365-4823-8692-3e3f5bde7849 None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:31.432738 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314ee23d-c365-4823-8692-3e3f5bde7849 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:31.432966 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314ee23d-c365-4823-8692-3e3f5bde7849 None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:31.434753 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 486/968] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:35:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:35:31.441251 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e08a75-51ff-4303-9e41-d1ff5ff45f6f None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:31.441539 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e08a75-51ff-4303-9e41-d1ff5ff45f6f None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:31.441803 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e08a75-51ff-4303-9e41-d1ff5ff45f6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:31.466138 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-63e08a75-51ff-4303-9e41-d1ff5ff45f6f None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:35:31.467438 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-63e08a75-51ff-4303-9e41-d1ff5ff45f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:35:31.509754 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d41735-8c23-4246-b3c0-c7d787eea07d None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:31.557943 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cdcf2c-8b0e-498c-963c-86b4727a8700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:31.558991 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cdcf2c-8b0e-498c-963c-86b4727a8700 None admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:31.559274 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cdcf2c-8b0e-498c-963c-86b4727a8700 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:31.559547 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cdcf2c-8b0e-498c-963c-86b4727a8700 None admin] PATH_INFO: `/v3/services` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:31.560793 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cdcf2c-8b0e-498c-963c-86b4727a8700 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:31.562277 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cdcf2c-8b0e-498c-963c-86b4727a8700 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:31.566585 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 487/969] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:35:31 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 08:35:31.631080 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 483/970] 10.222.0.55 () {60 vars in 1141 bytes} [Wed Aug 24 08:35:31 2022] POST /identity/v3/services => generated 242 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:35:31.842349 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e08a75-51ff-4303-9e41-d1ff5ff45f6f None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:31.906807 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:32.027857 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e08a75-51ff-4303-9e41-d1ff5ff45f6f None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:32.110799 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126af1bc-100f-4741-91b0-3694c4e56c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:32.111884 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126af1bc-100f-4741-91b0-3694c4e56c0e None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:32.112365 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126af1bc-100f-4741-91b0-3694c4e56c0e None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:32.112894 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126af1bc-100f-4741-91b0-3694c4e56c0e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:32.132934 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126af1bc-100f-4741-91b0-3694c4e56c0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:35:32.142962 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-126af1bc-100f-4741-91b0-3694c4e56c0e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 08:35:32.142962 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:32.142962 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:32.142962 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:32.142962 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:32.142962 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:32.142962 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:32.142962 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:32.142962 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:32.142962 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:32.142962 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:32.142962 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:32.142962 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:35:32.142962 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:35:32.142962 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:35:32.142962 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:35:32.144026 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:35:32.144026 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:35:32.144026 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:35:32.144026 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:35:32.144026 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:32.144026 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:32.144026 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:32.144026 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:32.144026 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:32.144026 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:32.144026 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:32.144026 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:32.144026 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:32.144026 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:32.144026 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:32.144026 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:32.145332 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:32.145332 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:32.145332 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:32.145332 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:32.145332 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:35:32.145332 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:35:32.145332 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:35:32.145332 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:35:32.145332 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:35:32.145332 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:35:32.145332 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 08:35:32.145332 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:35:32.146401 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 488/971] 10.222.0.55 () {56 vars in 1120 bytes} [Wed Aug 24 08:35:31 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:32.155285 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cdcf2c-8b0e-498c-963c-86b4727a8700 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:32.219144 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:32.355942 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cdcf2c-8b0e-498c-963c-86b4727a8700 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:32.414370 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80a414f-a77e-4f81-8ac9-6c2e12f16858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:32.415402 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80a414f-a77e-4f81-8ac9-6c2e12f16858 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:32.415885 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80a414f-a77e-4f81-8ac9-6c2e12f16858 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:32.416344 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80a414f-a77e-4f81-8ac9-6c2e12f16858 None admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:32.417528 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80a414f-a77e-4f81-8ac9-6c2e12f16858 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:32.418070 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80a414f-a77e-4f81-8ac9-6c2e12f16858 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:32.421235 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80a414f-a77e-4f81-8ac9-6c2e12f16858 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:32.440574 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 484/972] 10.222.0.55 () {56 vars in 1115 bytes} [Wed Aug 24 08:35:32 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:32.443031 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126af1bc-100f-4741-91b0-3694c4e56c0e None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:32.605750 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:32.621143 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126af1bc-100f-4741-91b0-3694c4e56c0e None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:32.689471 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675695e0-fb79-4707-9d97-c4793e5e53bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:32.690650 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675695e0-fb79-4707-9d97-c4793e5e53bb None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:32.691307 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675695e0-fb79-4707-9d97-c4793e5e53bb None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:32.691589 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675695e0-fb79-4707-9d97-c4793e5e53bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:32.711370 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675695e0-fb79-4707-9d97-c4793e5e53bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:32.714169 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675695e0-fb79-4707-9d97-c4793e5e53bb None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:32.743377 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-675695e0-fb79-4707-9d97-c4793e5e53bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:32.743377 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:32.743377 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:32.743377 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:32.743377 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:32.743377 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:32.743377 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:32.743377 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:32.743377 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:32.743377 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:32.743377 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:32.743377 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:32.743377 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:35:32.743377 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:35:32.743377 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:35:32.743377 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:35:32.745669 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:32.745669 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:32.745669 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:32.745669 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:32.745669 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:32.745669 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:32.745669 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:32.745669 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:32.745669 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:32.745669 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:32.745669 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:32.745669 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:32.745669 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:32.745669 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:32.745669 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:32.745669 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:32.745669 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:35:32.747421 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:35:32.747421 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:32.747421 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:35:32.747421 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 489/973] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:35:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:32.757807 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80a414f-a77e-4f81-8ac9-6c2e12f16858 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:32.936653 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:32.955487 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80a414f-a77e-4f81-8ac9-6c2e12f16858 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:33.022917 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bbb3bd-6341-4499-9e7f-8a02550b8204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:33.023950 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bbb3bd-6341-4499-9e7f-8a02550b8204 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:33.024395 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bbb3bd-6341-4499-9e7f-8a02550b8204 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:33.024883 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bbb3bd-6341-4499-9e7f-8a02550b8204 None admin] PATH_INFO: `/v3/domains` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:33.026019 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bbb3bd-6341-4499-9e7f-8a02550b8204 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:33.026532 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bbb3bd-6341-4499-9e7f-8a02550b8204 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:33.029051 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bbb3bd-6341-4499-9e7f-8a02550b8204 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:33.058173 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 485/974] 10.222.0.55 () {56 vars in 1109 bytes} [Wed Aug 24 08:35:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:33.062601 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675695e0-fb79-4707-9d97-c4793e5e53bb None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:33.265811 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675695e0-fb79-4707-9d97-c4793e5e53bb None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:33.333982 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3ba82c-5a16-4743-abac-2177a1e36f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:33.335126 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3ba82c-5a16-4743-abac-2177a1e36f56 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:33.335721 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3ba82c-5a16-4743-abac-2177a1e36f56 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:33.336221 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3ba82c-5a16-4743-abac-2177a1e36f56 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:33.351159 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3ba82c-5a16-4743-abac-2177a1e36f56 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:33.354014 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3ba82c-5a16-4743-abac-2177a1e36f56 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:33.377541 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-be3ba82c-5a16-4743-abac-2177a1e36f56 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 08:35:33.377541 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:33.377541 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:33.377541 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:33.377541 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:33.377541 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:33.377541 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:33.377541 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:33.377541 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:33.377541 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:33.377541 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:33.377541 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:33.377541 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:35:33.377541 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:35:33.377541 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:35:33.377541 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:35:33.378637 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:33.378637 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:33.378637 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:35:33.378637 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:35:33.378637 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:35:33.378637 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:35:33.378637 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:33.378637 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:33.378637 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:33.378637 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:33.378637 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:33.378637 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:33.378637 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:33.378637 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:33.378637 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:33.378637 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:33.378637 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:33.378637 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:33.379605 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:33.379605 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:33.379605 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:35:33.379605 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:35:33.379605 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:35:33.379605 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:35:33.379605 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:35:33.379605 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:35:33.379605 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 08:35:33.379605 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:35:33.385458 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 490/975] 10.222.0.55 () {56 vars in 1117 bytes} [Wed Aug 24 08:35:33 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:33.386638 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bbb3bd-6341-4499-9e7f-8a02550b8204 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:33.525948 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75436f14-0079-4ac1-8e1d-998cc9064685 None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:33.526154 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75436f14-0079-4ac1-8e1d-998cc9064685 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:33.526418 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75436f14-0079-4ac1-8e1d-998cc9064685 None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:33.527951 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 491/976] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:35:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:35:33.534171 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a07fb0-6379-4905-8f2b-68ba4857386e None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:33.534663 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a07fb0-6379-4905-8f2b-68ba4857386e None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:33.535118 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a07fb0-6379-4905-8f2b-68ba4857386e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:33.547590 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:33.567126 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-e1a07fb0-6379-4905-8f2b-68ba4857386e None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:35:33.568408 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-e1a07fb0-6379-4905-8f2b-68ba4857386e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:35:33.572875 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bbb3bd-6341-4499-9e7f-8a02550b8204 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:33.641970 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c563fc55-f5a4-456c-ad6c-422c333386a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:33.643026 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c563fc55-f5a4-456c-ad6c-422c333386a8 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:33.643621 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c563fc55-f5a4-456c-ad6c-422c333386a8 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:33.643996 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c563fc55-f5a4-456c-ad6c-422c333386a8 None admin] PATH_INFO: `/v3/users` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:33.645421 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c563fc55-f5a4-456c-ad6c-422c333386a8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:33.645962 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c563fc55-f5a4-456c-ad6c-422c333386a8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:33.648828 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c563fc55-f5a4-456c-ad6c-422c333386a8 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:33.653667 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 492/977] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:35:33 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:35:33.688635 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 486/978] 10.222.0.55 () {56 vars in 1149 bytes} [Wed Aug 24 08:35:33 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:33.697909 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a07fb0-6379-4905-8f2b-68ba4857386e None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:33.899291 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a07fb0-6379-4905-8f2b-68ba4857386e None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:33.943912 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c563fc55-f5a4-456c-ad6c-422c333386a8 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:33.956733 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:33.968028 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610ee957-f68e-4308-947c-b0ab9bc18d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:33.968674 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610ee957-f68e-4308-947c-b0ab9bc18d4c None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:33.969083 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610ee957-f68e-4308-947c-b0ab9bc18d4c None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:33.969691 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610ee957-f68e-4308-947c-b0ab9bc18d4c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:33.973399 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:34.057905 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610ee957-f68e-4308-947c-b0ab9bc18d4c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:34.061361 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610ee957-f68e-4308-947c-b0ab9bc18d4c None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:34.092019 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c563fc55-f5a4-456c-ad6c-422c333386a8 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:34.139910 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cafe00-7207-46da-8e3b-be1bb6d34a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:34.141044 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cafe00-7207-46da-8e3b-be1bb6d34a67 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:34.141539 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cafe00-7207-46da-8e3b-be1bb6d34a67 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:34.142026 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cafe00-7207-46da-8e3b-be1bb6d34a67 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:34.143083 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cafe00-7207-46da-8e3b-be1bb6d34a67 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:34.145729 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cafe00-7207-46da-8e3b-be1bb6d34a67 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:34.162056 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-78cafe00-7207-46da-8e3b-be1bb6d34a67 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 08:35:34.162056 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:34.162056 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 08:35:34.162056 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 08:35:34.162056 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 08:35:34.162056 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 08:35:34.162056 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 08:35:34.162056 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:35:34.162056 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 08:35:34.162056 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:35:34.162056 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:35:34.162056 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:35:34.162056 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:34.162056 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:34.162056 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:34.162056 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:34.162056 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:34.163287 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:34.163287 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:34.163287 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:34.163287 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:34.163287 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:34.163287 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:34.163287 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 08:35:34.163287 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 08:35:34.163287 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 08:35:34.163287 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 08:35:34.163287 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:34.163287 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:34.163287 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:34.163287 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:34.163287 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:34.163287 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:34.164989 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:34.164989 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:34.164989 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:34.164989 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:34.164989 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:34.164989 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:34.164989 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:34.164989 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:34.164989 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:34.164989 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:34.164989 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 08:35:34.164989 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:35:34.164989 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 08:35:34.164989 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:35:34.164989 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 487/979] 10.222.0.55 () {56 vars in 1126 bytes} [Wed Aug 24 08:35:33 2022] GET /identity/v3/services/object-store => generated 93 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:35:34.166199 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f874a381-5a7a-4c89-b1da-cc02edfe06ca None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:34.166658 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f874a381-5a7a-4c89-b1da-cc02edfe06ca None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:34.167152 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f874a381-5a7a-4c89-b1da-cc02edfe06ca None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:34.170461 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 488/980] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:35:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:35:34.171731 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f874a381-5a7a-4c89-b1da-cc02edfe06ca None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:34.185779 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 493/981] 10.222.0.55 () {58 vars in 1282 bytes} [Wed Aug 24 08:35:33 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 08:35:34.187066 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fec6f8-be5d-4909-abbe-0ecf23591682 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:34.187339 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fec6f8-be5d-4909-abbe-0ecf23591682 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:34.187609 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fec6f8-be5d-4909-abbe-0ecf23591682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:34.210952 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-e1fec6f8-be5d-4909-abbe-0ecf23591682 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:35:34.212172 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-e1fec6f8-be5d-4909-abbe-0ecf23591682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:35:34.243637 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:34.268867 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:34.291878 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 494/982] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:35:34 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:35:34.292871 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fec6f8-be5d-4909-abbe-0ecf23591682 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:34.320810 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f874a381-5a7a-4c89-b1da-cc02edfe06ca None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:34.372628 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f7b584-710f-4299-9935-5dc06b82e206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:34.374220 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f7b584-710f-4299-9935-5dc06b82e206 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:34.374726 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f7b584-710f-4299-9935-5dc06b82e206 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:34.375202 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f7b584-710f-4299-9935-5dc06b82e206 None admin] PATH_INFO: `/v3/services` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:34.376210 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f7b584-710f-4299-9935-5dc06b82e206 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:34.376783 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f7b584-710f-4299-9935-5dc06b82e206 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:34.379443 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f7b584-710f-4299-9935-5dc06b82e206 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:34.394052 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 489/983] 10.222.0.55 () {56 vars in 1122 bytes} [Wed Aug 24 08:35:34 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:34.396319 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f7b584-710f-4299-9935-5dc06b82e206 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:34.478856 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fec6f8-be5d-4909-abbe-0ecf23591682 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:34.535647 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f7b584-710f-4299-9935-5dc06b82e206 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:34.535853 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ad5f71-d50c-46a4-8ad4-3864120bcc65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:34.536831 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ad5f71-d50c-46a4-8ad4-3864120bcc65 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:34.537359 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ad5f71-d50c-46a4-8ad4-3864120bcc65 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:34.537872 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ad5f71-d50c-46a4-8ad4-3864120bcc65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:34.558148 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ad5f71-d50c-46a4-8ad4-3864120bcc65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:34.560220 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ad5f71-d50c-46a4-8ad4-3864120bcc65 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:34.577611 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188604ed-07e8-44cb-afde-710d30c56dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:34.578230 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188604ed-07e8-44cb-afde-710d30c56dbd None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:34.578508 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188604ed-07e8-44cb-afde-710d30c56dbd None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:34.578778 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188604ed-07e8-44cb-afde-710d30c56dbd None admin] PATH_INFO: `/v3/services` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:34.579417 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188604ed-07e8-44cb-afde-710d30c56dbd None admin] RBAC: Adding query filter params (type=object-store) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:34.579720 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188604ed-07e8-44cb-afde-710d30c56dbd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:34.581152 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188604ed-07e8-44cb-afde-710d30c56dbd None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:34.584101 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:34.585592 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-a4ad5f71-d50c-46a4-8ad4-3864120bcc65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:34.585592 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:34.585592 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:34.585592 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:34.585592 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:34.585592 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:34.585592 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:34.585592 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:34.585592 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:34.585592 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:34.585592 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:34.585592 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:34.585592 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:35:34.585592 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:35:34.585592 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:35:34.585592 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:35:34.586648 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:34.586648 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:34.586648 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:34.586648 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:34.586648 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:34.586648 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:34.586648 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:34.586648 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:34.586648 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:34.586648 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:34.586648 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:34.586648 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:34.586648 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:34.586648 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:34.586648 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:34.586648 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:34.586648 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:35:34.587555 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:35:34.587555 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:34.587555 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:35:34.587555 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 495/984] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:35:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:34.588226 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 490/985] 10.222.0.55 () {56 vars in 1122 bytes} [Wed Aug 24 08:35:34 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:34.588752 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ad5f71-d50c-46a4-8ad4-3864120bcc65 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:34.594099 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188604ed-07e8-44cb-afde-710d30c56dbd None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:34.621455 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:34.725901 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ad5f71-d50c-46a4-8ad4-3864120bcc65 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:34.756089 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188604ed-07e8-44cb-afde-710d30c56dbd None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:34.790076 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278287d2-58ec-4197-ad69-502cd9aa0856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:34.791219 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278287d2-58ec-4197-ad69-502cd9aa0856 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:34.791630 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278287d2-58ec-4197-ad69-502cd9aa0856 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:34.792103 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278287d2-58ec-4197-ad69-502cd9aa0856 None admin] PATH_INFO: `/v3/services/compute` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:34.793241 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278287d2-58ec-4197-ad69-502cd9aa0856 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:34.796515 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278287d2-58ec-4197-ad69-502cd9aa0856 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:34.810439 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8f3828-f95d-4473-aa41-87edd469fc5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:34.811486 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8f3828-f95d-4473-aa41-87edd469fc5b None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:34.812014 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8f3828-f95d-4473-aa41-87edd469fc5b None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:34.812514 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8f3828-f95d-4473-aa41-87edd469fc5b None admin] PATH_INFO: `/v3/domains` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:34.813667 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8f3828-f95d-4473-aa41-87edd469fc5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:34.814217 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8f3828-f95d-4473-aa41-87edd469fc5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:34.814557 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-278287d2-58ec-4197-ad69-502cd9aa0856 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 08:35:34.814557 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:34.814557 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 08:35:34.814557 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 08:35:34.814557 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 08:35:34.814557 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 08:35:34.814557 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 08:35:34.814557 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:35:34.814557 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 08:35:34.814557 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:35:34.814557 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:35:34.814557 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:35:34.814557 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:34.814557 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:34.814557 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:34.814557 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:34.814557 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:34.815596 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:34.815596 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:34.815596 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:34.815596 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:34.815596 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:34.815596 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:34.815596 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 08:35:34.815596 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 08:35:34.815596 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 08:35:34.815596 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 08:35:34.815596 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:34.815596 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:34.815596 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:34.815596 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:34.815596 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:34.815596 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:34.816473 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:34.816473 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:34.816473 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:34.816473 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:34.816473 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:34.816473 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:34.816473 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:34.816473 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:34.816473 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:34.816473 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:34.816473 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 08:35:34.816473 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:35:34.816473 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 08:35:34.816473 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:35:34.817563 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8f3828-f95d-4473-aa41-87edd469fc5b None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:34.817638 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 496/986] 10.222.0.55 () {56 vars in 1111 bytes} [Wed Aug 24 08:35:34 2022] GET /identity/v3/services/compute => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:34.818065 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278287d2-58ec-4197-ad69-502cd9aa0856 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:34.837448 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 491/987] 10.222.0.55 () {56 vars in 1109 bytes} [Wed Aug 24 08:35:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:35:34.838551 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8f3828-f95d-4473-aa41-87edd469fc5b None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:34.926386 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:34.948381 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:34.970461 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278287d2-58ec-4197-ad69-502cd9aa0856 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:34.984561 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8f3828-f95d-4473-aa41-87edd469fc5b None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:35.009472 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27da0054-3b95-422c-80ff-8616e826d62a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:35.010098 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27da0054-3b95-422c-80ff-8616e826d62a None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:35.010447 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27da0054-3b95-422c-80ff-8616e826d62a None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:35.010796 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27da0054-3b95-422c-80ff-8616e826d62a None admin] PATH_INFO: `/v3/endpoints` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:35.011606 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27da0054-3b95-422c-80ff-8616e826d62a None admin] RBAC: Adding query filter params (interface=admin, service_id=90a2f3f416014911aeb8a2aef87caf09, region_id=RegionOne) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:35.012006 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27da0054-3b95-422c-80ff-8616e826d62a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:35.014081 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27da0054-3b95-422c-80ff-8616e826d62a None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:35.020052 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 497/988] 10.222.0.55 () {56 vars in 1249 bytes} [Wed Aug 24 08:35:34 2022] GET /identity/v3/endpoints?service_id=90a2f3f416014911aeb8a2aef87caf09&interface=admin®ion_id=RegionOne => generated 193 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:35.021147 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27da0054-3b95-422c-80ff-8616e826d62a None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:35.042161 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568c7b70-5542-4186-ba3a-907f58907a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:35.043214 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568c7b70-5542-4186-ba3a-907f58907a9e None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:35.043611 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568c7b70-5542-4186-ba3a-907f58907a9e None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:35.043937 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568c7b70-5542-4186-ba3a-907f58907a9e None admin] PATH_INFO: `/v3/services` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:35.044877 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568c7b70-5542-4186-ba3a-907f58907a9e None admin] RBAC: Adding query filter params (name=compute) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:35.045381 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568c7b70-5542-4186-ba3a-907f58907a9e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:35.047867 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568c7b70-5542-4186-ba3a-907f58907a9e None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:35.060850 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 492/989] 10.222.0.55 () {56 vars in 1112 bytes} [Wed Aug 24 08:35:34 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:35.060850 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568c7b70-5542-4186-ba3a-907f58907a9e None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:35.130689 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27da0054-3b95-422c-80ff-8616e826d62a None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:35.168782 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03deb236-9aec-4fef-bc52-936543a00774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:35.169432 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03deb236-9aec-4fef-bc52-936543a00774 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:35.169713 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03deb236-9aec-4fef-bc52-936543a00774 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:35.169985 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03deb236-9aec-4fef-bc52-936543a00774 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:35.177144 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03deb236-9aec-4fef-bc52-936543a00774 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:35.178613 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03deb236-9aec-4fef-bc52-936543a00774 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:35.190928 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-03deb236-9aec-4fef-bc52-936543a00774 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:35:35.190928 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:35.190928 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:35.190928 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:35.190928 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:35.190928 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:35.190928 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:35.190928 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:35.190928 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:35.190928 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:35.190928 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:35.190928 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:35.190928 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:35:35.190928 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:35:35.190928 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:35:35.192451 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:35:35.192451 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:35.192451 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:35.192451 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:35.192451 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:35.192451 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:35.192451 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:35.192451 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:35.192451 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:35.192451 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:35.192451 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:35.192451 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:35.192451 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:35.192451 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:35.192451 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:35.192451 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:35.192451 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:35.193649 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:35:35.193649 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:35:35.193649 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:35:35.193649 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:35:35.193649 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:35:35.193649 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:35:35.193649 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:35:35.193649 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:35:35.194086 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568c7b70-5542-4186-ba3a-907f58907a9e None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:35.194172 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 498/990] 10.222.0.55 () {56 vars in 1111 bytes} [Wed Aug 24 08:35:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:35.194172 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03deb236-9aec-4fef-bc52-936543a00774 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:35.198471 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:35.225571 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:35.236945 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230fc3bc-d65f-4751-bab2-87c9df3e2160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:35.237562 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230fc3bc-d65f-4751-bab2-87c9df3e2160 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:35.237843 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230fc3bc-d65f-4751-bab2-87c9df3e2160 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:35.238123 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230fc3bc-d65f-4751-bab2-87c9df3e2160 None admin] PATH_INFO: `/v3/services` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:35.238736 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230fc3bc-d65f-4751-bab2-87c9df3e2160 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:35.240104 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230fc3bc-d65f-4751-bab2-87c9df3e2160 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:35.246738 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 493/991] 10.222.0.55 () {56 vars in 1087 bytes} [Wed Aug 24 08:35:35 2022] GET /identity/v3/services => generated 1727 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:35:35.248125 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230fc3bc-d65f-4751-bab2-87c9df3e2160 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:35.353578 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03deb236-9aec-4fef-bc52-936543a00774 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:35.396186 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230fc3bc-d65f-4751-bab2-87c9df3e2160 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:35.402907 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142b58f6-3c88-4b7a-afd5-1eab4a2547ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:35.402907 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142b58f6-3c88-4b7a-afd5-1eab4a2547ec None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:35.403335 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142b58f6-3c88-4b7a-afd5-1eab4a2547ec None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:35.403469 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142b58f6-3c88-4b7a-afd5-1eab4a2547ec None admin] PATH_INFO: `/v3/services` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:35.404143 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142b58f6-3c88-4b7a-afd5-1eab4a2547ec None admin] RBAC: Adding query filter params (type=compute) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:35.404452 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142b58f6-3c88-4b7a-afd5-1eab4a2547ec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:35.405944 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142b58f6-3c88-4b7a-afd5-1eab4a2547ec None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:35.412213 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 499/992] 10.222.0.55 () {56 vars in 1112 bytes} [Wed Aug 24 08:35:35 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:35.419462 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142b58f6-3c88-4b7a-afd5-1eab4a2547ec None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:35.448828 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a900d4f-1e1b-480d-a030-c793dcd70671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:35.449886 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a900d4f-1e1b-480d-a030-c793dcd70671 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:35.450415 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a900d4f-1e1b-480d-a030-c793dcd70671 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:35.451455 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a900d4f-1e1b-480d-a030-c793dcd70671 None admin] PATH_INFO: `/v3/projects` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:35.452821 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a900d4f-1e1b-480d-a030-c793dcd70671 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:35.453370 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a900d4f-1e1b-480d-a030-c793dcd70671 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:35.456013 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a900d4f-1e1b-480d-a030-c793dcd70671 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:35.474067 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 494/993] 10.222.0.55 () {56 vars in 1148 bytes} [Wed Aug 24 08:35:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:35.480096 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a900d4f-1e1b-480d-a030-c793dcd70671 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:35.539228 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:35.547358 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:35.608419 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a900d4f-1e1b-480d-a030-c793dcd70671 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:35.611077 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142b58f6-3c88-4b7a-afd5-1eab4a2547ec None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:35.649754 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fef90fc-b844-4f40-9f0d-042609071c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:35.650391 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fef90fc-b844-4f40-9f0d-042609071c4f None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:35.650703 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fef90fc-b844-4f40-9f0d-042609071c4f None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:35.651427 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fef90fc-b844-4f40-9f0d-042609071c4f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:35.651695 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fef90fc-b844-4f40-9f0d-042609071c4f None admin] RBAC: Adding query filter params (role.id=371e1f8b73f144aea9bb7d20d253e5a0, scope.project.id=420b18725f2342159646fb86438318fd, user.id=f652a368b1cc42ae8b6f6c7a6dff6f2f) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:35.651968 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fef90fc-b844-4f40-9f0d-042609071c4f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:35.653447 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fef90fc-b844-4f40-9f0d-042609071c4f None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:35.672587 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 495/994] 10.222.0.55 () {56 vars in 1374 bytes} [Wed Aug 24 08:35:35 2022] GET /identity/v3/role_assignments?user.id=f652a368b1cc42ae8b6f6c7a6dff6f2f&scope.project.id=420b18725f2342159646fb86438318fd&role.id=371e1f8b73f144aea9bb7d20d253e5a0 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:35:35.672867 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab49bd8b-aa3a-4b7c-a76e-45b41a5550fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:35.672867 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab49bd8b-aa3a-4b7c-a76e-45b41a5550fe None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:35.672867 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab49bd8b-aa3a-4b7c-a76e-45b41a5550fe None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:35.672867 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab49bd8b-aa3a-4b7c-a76e-45b41a5550fe None admin] PATH_INFO: `/v3/endpoints` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:35.673471 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab49bd8b-aa3a-4b7c-a76e-45b41a5550fe None admin] RBAC: Adding query filter params (interface=public, service_id=63ff6582b0b54aacbd5df60be69d528a, region_id=RegionOne) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:35.673471 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab49bd8b-aa3a-4b7c-a76e-45b41a5550fe None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:35.673471 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab49bd8b-aa3a-4b7c-a76e-45b41a5550fe None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:35.680977 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 500/995] 10.222.0.55 () {56 vars in 1251 bytes} [Wed Aug 24 08:35:35 2022] GET /identity/v3/endpoints?service_id=63ff6582b0b54aacbd5df60be69d528a&interface=public®ion_id=RegionOne => generated 194 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:35:35.685891 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fef90fc-b844-4f40-9f0d-042609071c4f None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:35.819299 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fef90fc-b844-4f40-9f0d-042609071c4f None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:35.834637 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:35.861522 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4155e77e-0146-4bdb-b877-044a38ba46b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:35.862633 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4155e77e-0146-4bdb-b877-044a38ba46b9 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:35.863059 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4155e77e-0146-4bdb-b877-044a38ba46b9 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:35.863471 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4155e77e-0146-4bdb-b877-044a38ba46b9 None admin] PATH_INFO: `/v3/services` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:35.864231 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4155e77e-0146-4bdb-b877-044a38ba46b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:35.865799 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4155e77e-0146-4bdb-b877-044a38ba46b9 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:35.880576 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 496/996] 10.222.0.55 () {56 vars in 1087 bytes} [Wed Aug 24 08:35:35 2022] GET /identity/v3/services => generated 1727 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 08:35:37.259250 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7385416-256b-40e1-8afc-ed8a755eb9ed None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:37.260105 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7385416-256b-40e1-8afc-ed8a755eb9ed None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:37.260105 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7385416-256b-40e1-8afc-ed8a755eb9ed None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:37.264582 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 501/997] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:35:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:35:37.272310 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0684f9b-8ef0-4e9d-8cf7-15bf6617619a None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:37.272872 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0684f9b-8ef0-4e9d-8cf7-15bf6617619a None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:37.273328 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0684f9b-8ef0-4e9d-8cf7-15bf6617619a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:37.305056 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-f0684f9b-8ef0-4e9d-8cf7-15bf6617619a None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:35:37.307183 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-f0684f9b-8ef0-4e9d-8cf7-15bf6617619a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:35:37.431419 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 497/998] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:35:37 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:35:37.927728 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7385416-256b-40e1-8afc-ed8a755eb9ed None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:37.941976 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b060088d-f42c-4343-8ef1-b697ea03f093 None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:37.942306 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b060088d-f42c-4343-8ef1-b697ea03f093 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:37.942522 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b060088d-f42c-4343-8ef1-b697ea03f093 None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:37.943874 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 498/999] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:35:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:35:37.949764 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23333ffb-fc13-44a9-bc5c-e0fb7868f4f9 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:37.950103 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23333ffb-fc13-44a9-bc5c-e0fb7868f4f9 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:37.950470 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23333ffb-fc13-44a9-bc5c-e0fb7868f4f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:37.971134 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-23333ffb-fc13-44a9-bc5c-e0fb7868f4f9 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:35:37.972335 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-23333ffb-fc13-44a9-bc5c-e0fb7868f4f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:35:38.069110 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7385416-256b-40e1-8afc-ed8a755eb9ed None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:38.096593 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 499/1000] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:35:37 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 08:35:38.096593 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6ddf2d-907a-4221-8928-1b42799df34e None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:38.096593 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6ddf2d-907a-4221-8928-1b42799df34e None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:38.096593 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6ddf2d-907a-4221-8928-1b42799df34e None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:38.096593 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 500/1001] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:35:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:35:38.100129 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:38.103316 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03871309-2004-4226-a60e-7115537826c5 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:38.103834 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03871309-2004-4226-a60e-7115537826c5 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:38.104337 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03871309-2004-4226-a60e-7115537826c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:38.104445 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d34d560-e2ed-44e8-b3a6-4c9ebeeab501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:38.105018 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d34d560-e2ed-44e8-b3a6-4c9ebeeab501 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:38.105301 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d34d560-e2ed-44e8-b3a6-4c9ebeeab501 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:38.105571 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d34d560-e2ed-44e8-b3a6-4c9ebeeab501 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:38.106153 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d34d560-e2ed-44e8-b3a6-4c9ebeeab501 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:38.108051 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d34d560-e2ed-44e8-b3a6-4c9ebeeab501 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:38.118061 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-5d34d560-e2ed-44e8-b3a6-4c9ebeeab501 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 08:35:38.118061 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:38.118061 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 08:35:38.118061 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 08:35:38.118061 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 08:35:38.118061 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 08:35:38.118061 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 08:35:38.118061 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:35:38.118061 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 08:35:38.118061 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:35:38.118061 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:35:38.118061 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:35:38.118061 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:38.118061 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:38.118061 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:38.118061 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:38.118061 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:38.122420 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:38.122420 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:38.122420 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:38.122420 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:38.122420 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:38.122420 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:38.122420 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 08:35:38.122420 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 08:35:38.122420 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 08:35:38.122420 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 08:35:38.122420 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:38.122420 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:38.122420 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:38.122420 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:38.122420 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:38.122420 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:38.124198 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:38.124198 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:38.124198 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:38.124198 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:38.124198 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:38.124198 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:38.124198 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:38.124198 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:38.124198 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:38.124198 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:38.124198 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 08:35:38.124198 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:35:38.124198 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 08:35:38.124198 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:35:38.124198 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 502/1002] 10.222.0.55 () {56 vars in 1126 bytes} [Wed Aug 24 08:35:37 2022] GET /identity/v3/services/object-store => generated 93 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:38.131430 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d34d560-e2ed-44e8-b3a6-4c9ebeeab501 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:38.140357 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-03871309-2004-4226-a60e-7115537826c5 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:35:38.142540 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-03871309-2004-4226-a60e-7115537826c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:35:38.187496 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:38.250987 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 501/1003] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:35:38 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 08:35:38.314970 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d34d560-e2ed-44e8-b3a6-4c9ebeeab501 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:38.365341 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0cb099-df7a-4d7f-86fc-05670297ff0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:38.365981 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0cb099-df7a-4d7f-86fc-05670297ff0c None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:38.366266 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0cb099-df7a-4d7f-86fc-05670297ff0c None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:38.366544 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0cb099-df7a-4d7f-86fc-05670297ff0c None admin] PATH_INFO: `/v3/services` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:38.367364 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0cb099-df7a-4d7f-86fc-05670297ff0c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:38.367676 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0cb099-df7a-4d7f-86fc-05670297ff0c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:38.368998 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03871309-2004-4226-a60e-7115537826c5 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:38.369725 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0cb099-df7a-4d7f-86fc-05670297ff0c None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:38.384154 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 503/1004] 10.222.0.55 () {56 vars in 1122 bytes} [Wed Aug 24 08:35:38 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:38.386668 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0cb099-df7a-4d7f-86fc-05670297ff0c None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:38.498815 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03871309-2004-4226-a60e-7115537826c5 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:38.499023 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:38.543563 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62582020-adca-4e0c-a6be-ef55a80d77bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:38.545180 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62582020-adca-4e0c-a6be-ef55a80d77bf None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:38.545512 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62582020-adca-4e0c-a6be-ef55a80d77bf None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:38.545983 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62582020-adca-4e0c-a6be-ef55a80d77bf None admin] PATH_INFO: `/v3/services/compute` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:38.546813 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62582020-adca-4e0c-a6be-ef55a80d77bf None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:38.547669 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0cb099-df7a-4d7f-86fc-05670297ff0c None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:38.548756 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62582020-adca-4e0c-a6be-ef55a80d77bf None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:38.562528 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-62582020-adca-4e0c-a6be-ef55a80d77bf None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 08:35:38.562528 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:38.562528 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 08:35:38.562528 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 08:35:38.562528 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 08:35:38.562528 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 08:35:38.562528 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 08:35:38.562528 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:35:38.562528 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 08:35:38.562528 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:35:38.562528 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:35:38.562528 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:35:38.562528 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:38.562528 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:38.562528 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:38.562528 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:38.562528 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:38.563838 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:38.563838 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:38.563838 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:38.563838 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:38.563838 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:38.563838 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:38.563838 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 08:35:38.563838 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 08:35:38.563838 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 08:35:38.563838 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 08:35:38.563838 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:38.563838 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:38.563838 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:38.563838 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:38.563838 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:38.563838 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:38.564803 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:38.564803 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:38.564803 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:38.564803 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:38.564803 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:38.564803 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:38.564803 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:38.564803 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:38.564803 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:38.564803 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:38.564803 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 08:35:38.564803 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:35:38.564803 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 08:35:38.564803 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:35:38.564803 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 502/1005] 10.222.0.55 () {56 vars in 1111 bytes} [Wed Aug 24 08:35:38 2022] GET /identity/v3/services/compute => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:38.565650 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62582020-adca-4e0c-a6be-ef55a80d77bf None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:38.591017 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb9db1a-f706-4880-aab1-d1705e1b498e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:38.592183 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb9db1a-f706-4880-aab1-d1705e1b498e None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:38.592461 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb9db1a-f706-4880-aab1-d1705e1b498e None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:38.592771 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb9db1a-f706-4880-aab1-d1705e1b498e None admin] PATH_INFO: `/v3/services` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:38.593417 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb9db1a-f706-4880-aab1-d1705e1b498e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:38.593716 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb9db1a-f706-4880-aab1-d1705e1b498e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:38.595099 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb9db1a-f706-4880-aab1-d1705e1b498e None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:38.600496 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 504/1006] 10.222.0.55 () {56 vars in 1122 bytes} [Wed Aug 24 08:35:38 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:38.601547 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb9db1a-f706-4880-aab1-d1705e1b498e None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:38.617675 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:38.709691 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62582020-adca-4e0c-a6be-ef55a80d77bf None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:38.718019 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb9db1a-f706-4880-aab1-d1705e1b498e None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:38.760891 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:38.761392 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2dff77-2ebc-45a2-9581-d342a86b8f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:38.762062 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2dff77-2ebc-45a2-9581-d342a86b8f89 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:38.762331 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2dff77-2ebc-45a2-9581-d342a86b8f89 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:38.762603 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2dff77-2ebc-45a2-9581-d342a86b8f89 None admin] PATH_INFO: `/v3/services/image` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:38.763706 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2dff77-2ebc-45a2-9581-d342a86b8f89 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:38.765151 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2dff77-2ebc-45a2-9581-d342a86b8f89 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:38.774585 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-1d2dff77-2ebc-45a2-9581-d342a86b8f89 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 08:35:38.774585 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:38.774585 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 08:35:38.774585 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 08:35:38.774585 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 08:35:38.774585 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 08:35:38.774585 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 08:35:38.774585 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:35:38.774585 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 08:35:38.774585 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:35:38.774585 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:35:38.774585 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:35:38.774585 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:38.774585 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:38.774585 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:38.774585 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:38.774585 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:38.783441 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:38.783441 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:38.783441 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:38.783441 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:38.783441 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:38.783441 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:38.783441 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 08:35:38.783441 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 08:35:38.783441 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 08:35:38.783441 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 08:35:38.783441 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:38.783441 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:38.783441 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:38.783441 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:38.783441 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:38.783441 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:38.784382 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:38.784382 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:38.784382 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:38.784382 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:38.784382 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:38.784382 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:38.784382 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:38.784382 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:38.784382 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:38.784382 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:38.784382 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 08:35:38.784382 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:35:38.784382 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 08:35:38.784382 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:35:38.784382 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 503/1007] 10.222.0.55 () {56 vars in 1105 bytes} [Wed Aug 24 08:35:38 2022] GET /identity/v3/services/image => generated 86 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:38.785315 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2dff77-2ebc-45a2-9581-d342a86b8f89 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:38.786638 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac75940-efb5-4e07-9278-4b46cfad1a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:38.787562 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac75940-efb5-4e07-9278-4b46cfad1a60 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:38.787989 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac75940-efb5-4e07-9278-4b46cfad1a60 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:38.789054 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac75940-efb5-4e07-9278-4b46cfad1a60 None admin] PATH_INFO: `/v3/services` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:38.789502 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac75940-efb5-4e07-9278-4b46cfad1a60 None admin] RBAC: Adding query filter params (name=compute) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:38.789970 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac75940-efb5-4e07-9278-4b46cfad1a60 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:38.792407 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac75940-efb5-4e07-9278-4b46cfad1a60 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:38.808152 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 505/1008] 10.222.0.55 () {56 vars in 1112 bytes} [Wed Aug 24 08:35:38 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:38.808152 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac75940-efb5-4e07-9278-4b46cfad1a60 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:38.893051 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:38.900035 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2dff77-2ebc-45a2-9581-d342a86b8f89 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:38.941961 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281645c9-94a8-4be0-897f-c5fe8b7bbbeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:38.942194 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac75940-efb5-4e07-9278-4b46cfad1a60 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:38.943008 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281645c9-94a8-4be0-897f-c5fe8b7bbbeb None admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:38.943503 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281645c9-94a8-4be0-897f-c5fe8b7bbbeb None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:38.943983 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281645c9-94a8-4be0-897f-c5fe8b7bbbeb None admin] PATH_INFO: `/v3/endpoints` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:38.945234 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281645c9-94a8-4be0-897f-c5fe8b7bbbeb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:38.947403 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281645c9-94a8-4be0-897f-c5fe8b7bbbeb None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:38.979346 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c348458-b3e8-4d94-b514-e93c334df5eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:38.979973 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c348458-b3e8-4d94-b514-e93c334df5eb None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:38.980242 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c348458-b3e8-4d94-b514-e93c334df5eb None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:38.980508 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c348458-b3e8-4d94-b514-e93c334df5eb None admin] PATH_INFO: `/v3/services` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:38.981175 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c348458-b3e8-4d94-b514-e93c334df5eb None admin] RBAC: Adding query filter params (name=image) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:38.981477 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c348458-b3e8-4d94-b514-e93c334df5eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:38.982878 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c348458-b3e8-4d94-b514-e93c334df5eb None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:38.988115 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 506/1009] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:35:38 2022] GET /identity/v3/services?name=image => generated 122 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:38.989067 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c348458-b3e8-4d94-b514-e93c334df5eb None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:39.045357 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:39.059467 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 504/1010] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 08:35:38 2022] POST /identity/v3/endpoints => generated 325 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:35:39.060804 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281645c9-94a8-4be0-897f-c5fe8b7bbbeb None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:39.116813 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c348458-b3e8-4d94-b514-e93c334df5eb None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:39.182420 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643035a1-7a0a-4f5e-8ae4-26530d52aa6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:39.183157 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643035a1-7a0a-4f5e-8ae4-26530d52aa6e None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:39.183467 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643035a1-7a0a-4f5e-8ae4-26530d52aa6e None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:39.183739 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643035a1-7a0a-4f5e-8ae4-26530d52aa6e None admin] PATH_INFO: `/v3/services` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:39.184446 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643035a1-7a0a-4f5e-8ae4-26530d52aa6e None admin] RBAC: Adding query filter params (type=compute) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:39.184786 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643035a1-7a0a-4f5e-8ae4-26530d52aa6e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:39.186865 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643035a1-7a0a-4f5e-8ae4-26530d52aa6e None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:39.195576 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 507/1011] 10.222.0.55 () {56 vars in 1112 bytes} [Wed Aug 24 08:35:38 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:39.201818 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643035a1-7a0a-4f5e-8ae4-26530d52aa6e None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:39.210558 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281645c9-94a8-4be0-897f-c5fe8b7bbbeb None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:39.256779 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db7f531-f1ab-4d5f-b860-63671151e0b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:39.257693 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db7f531-f1ab-4d5f-b860-63671151e0b6 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:39.257984 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db7f531-f1ab-4d5f-b860-63671151e0b6 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:39.258246 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db7f531-f1ab-4d5f-b860-63671151e0b6 None admin] PATH_INFO: `/v3/services` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:39.258913 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db7f531-f1ab-4d5f-b860-63671151e0b6 None admin] RBAC: Adding query filter params (type=image) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:39.259190 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db7f531-f1ab-4d5f-b860-63671151e0b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:39.261415 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db7f531-f1ab-4d5f-b860-63671151e0b6 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:39.270688 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 505/1012] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:35:39 2022] GET /identity/v3/services?type=image => generated 122 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:39.356173 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643035a1-7a0a-4f5e-8ae4-26530d52aa6e None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:39.386612 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:39.426414 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abb40c2-fc37-4298-88b7-2f7b86be4269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:39.427271 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abb40c2-fc37-4298-88b7-2f7b86be4269 None admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:39.427700 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abb40c2-fc37-4298-88b7-2f7b86be4269 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:39.428188 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abb40c2-fc37-4298-88b7-2f7b86be4269 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:39.429530 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abb40c2-fc37-4298-88b7-2f7b86be4269 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:39.433188 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abb40c2-fc37-4298-88b7-2f7b86be4269 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:39.528710 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 508/1013] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 08:35:39 2022] POST /identity/v3/endpoints => generated 334 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:35:41.093361 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174a2096-3907-4da4-9479-9071e5638a35 None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:41.094261 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174a2096-3907-4da4-9479-9071e5638a35 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:41.094261 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174a2096-3907-4da4-9479-9071e5638a35 None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:41.095882 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 506/1014] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:35:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:35:41.101827 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e67217-1ad2-4164-9724-b428b9c48e6c None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:41.102692 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e67217-1ad2-4164-9724-b428b9c48e6c None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:41.102692 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e67217-1ad2-4164-9724-b428b9c48e6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:41.123162 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-51e67217-1ad2-4164-9724-b428b9c48e6c None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:35:41.124413 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-51e67217-1ad2-4164-9724-b428b9c48e6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:35:41.236608 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 509/1015] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:35:41 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:35:41.517601 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174a2096-3907-4da4-9479-9071e5638a35 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:41.541826 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:41.643728 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174a2096-3907-4da4-9479-9071e5638a35 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:41.680672 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de8a5ad-f708-4bdf-a2ac-a5edbcb35862 None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:41.680971 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de8a5ad-f708-4bdf-a2ac-a5edbcb35862 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:41.681300 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de8a5ad-f708-4bdf-a2ac-a5edbcb35862 None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:41.682582 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 510/1016] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:35:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:35:41.687490 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275aec92-4e8a-43e7-9231-d9fdb59d673c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:41.688258 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275aec92-4e8a-43e7-9231-d9fdb59d673c None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:41.688751 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275aec92-4e8a-43e7-9231-d9fdb59d673c None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:41.688951 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960a45c4-93b6-4c91-967a-0faf3bb27652 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:41.689226 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960a45c4-93b6-4c91-967a-0faf3bb27652 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:41.689382 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275aec92-4e8a-43e7-9231-d9fdb59d673c None admin] PATH_INFO: `/v3/domains/default` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:41.689522 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960a45c4-93b6-4c91-967a-0faf3bb27652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:41.690719 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275aec92-4e8a-43e7-9231-d9fdb59d673c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:41.692370 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275aec92-4e8a-43e7-9231-d9fdb59d673c None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:41.694583 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 507/1017] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:35:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:35:41.700253 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275aec92-4e8a-43e7-9231-d9fdb59d673c None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:41.711353 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-960a45c4-93b6-4c91-967a-0faf3bb27652 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:35:41.712982 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-960a45c4-93b6-4c91-967a-0faf3bb27652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:35:41.799665 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 511/1018] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:35:41 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 08:35:41.809921 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:41.828596 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275aec92-4e8a-43e7-9231-d9fdb59d673c None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:41.868096 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5fe97e-5635-4a69-beec-e8ffeb54d24f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:41.868938 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5fe97e-5635-4a69-beec-e8ffeb54d24f None admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:41.869367 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5fe97e-5635-4a69-beec-e8ffeb54d24f None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:41.869820 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5fe97e-5635-4a69-beec-e8ffeb54d24f None admin] PATH_INFO: `/v3/projects` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:41.870585 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5fe97e-5635-4a69-beec-e8ffeb54d24f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:41.872213 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5fe97e-5635-4a69-beec-e8ffeb54d24f None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:41.962363 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 508/1019] 10.222.0.55 () {60 vars in 1141 bytes} [Wed Aug 24 08:35:41 2022] POST /identity/v3/projects => generated 311 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:35:42.085191 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960a45c4-93b6-4c91-967a-0faf3bb27652 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:42.117605 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:42.229758 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960a45c4-93b6-4c91-967a-0faf3bb27652 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:42.272218 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1957ffe4-339a-4911-b4f8-71e2436df49b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:42.273158 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1957ffe4-339a-4911-b4f8-71e2436df49b None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:42.273587 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1957ffe4-339a-4911-b4f8-71e2436df49b None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:42.273996 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1957ffe4-339a-4911-b4f8-71e2436df49b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:42.287765 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1957ffe4-339a-4911-b4f8-71e2436df49b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:35:42.293132 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-1957ffe4-339a-4911-b4f8-71e2436df49b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 08:35:42.293132 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:42.293132 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:42.293132 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:42.293132 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:42.293132 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:42.293132 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:42.293132 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:42.293132 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:42.293132 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:42.293132 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:42.293132 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:42.293132 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:35:42.293132 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:35:42.293132 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:35:42.293132 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:35:42.295594 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:35:42.295594 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:35:42.295594 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:35:42.295594 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:35:42.295594 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:42.295594 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:42.295594 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:42.295594 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:42.295594 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:42.295594 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:42.295594 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:42.295594 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:42.295594 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:42.295594 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:42.295594 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:42.295594 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:42.297910 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:42.297910 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:42.297910 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:42.297910 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:42.297910 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:35:42.297910 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:35:42.297910 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:35:42.297910 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:35:42.297910 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:35:42.297910 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:35:42.297910 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 08:35:42.297910 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:35:42.297910 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 512/1020] 10.222.0.55 () {56 vars in 1120 bytes} [Wed Aug 24 08:35:42 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:35:42.302501 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5fe97e-5635-4a69-beec-e8ffeb54d24f None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:42.462012 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5fe97e-5635-4a69-beec-e8ffeb54d24f None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:42.503079 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beba3f4-06d1-49db-9b52-a07dd45d6095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:42.503701 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beba3f4-06d1-49db-9b52-a07dd45d6095 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:42.503973 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beba3f4-06d1-49db-9b52-a07dd45d6095 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:42.504240 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beba3f4-06d1-49db-9b52-a07dd45d6095 None admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:42.504963 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4beba3f4-06d1-49db-9b52-a07dd45d6095 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:42.505263 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4beba3f4-06d1-49db-9b52-a07dd45d6095 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:42.506661 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4beba3f4-06d1-49db-9b52-a07dd45d6095 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:42.513842 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:42.519284 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 509/1021] 10.222.0.55 () {56 vars in 1115 bytes} [Wed Aug 24 08:35:42 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:42.526614 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1957ffe4-339a-4911-b4f8-71e2436df49b None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:42.531333 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055b502f-a1c6-43d4-9908-97f4a473ff5b None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:42.531333 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055b502f-a1c6-43d4-9908-97f4a473ff5b None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:42.531333 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055b502f-a1c6-43d4-9908-97f4a473ff5b None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:42.534284 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 510/1022] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:35:42 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:35:42.540623 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eff2f5-2686-4eae-a893-631544f8da6b None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:42.541136 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eff2f5-2686-4eae-a893-631544f8da6b None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:42.541635 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eff2f5-2686-4eae-a893-631544f8da6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:42.573771 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-98eff2f5-2686-4eae-a893-631544f8da6b None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:35:42.574997 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-98eff2f5-2686-4eae-a893-631544f8da6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:35:42.689454 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:42.697681 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 511/1023] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:35:42 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 08:35:42.722038 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1957ffe4-339a-4911-b4f8-71e2436df49b None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:42.796953 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fd5e95-3927-4b53-9cda-d1ba3c73f2ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:42.798063 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fd5e95-3927-4b53-9cda-d1ba3c73f2ef None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:42.798556 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fd5e95-3927-4b53-9cda-d1ba3c73f2ef None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:42.799046 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fd5e95-3927-4b53-9cda-d1ba3c73f2ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:42.820773 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fd5e95-3927-4b53-9cda-d1ba3c73f2ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:42.823525 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fd5e95-3927-4b53-9cda-d1ba3c73f2ef None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:42.858442 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-15fd5e95-3927-4b53-9cda-d1ba3c73f2ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:42.858442 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:42.858442 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:42.858442 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:42.858442 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:42.858442 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:42.858442 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:42.858442 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:42.858442 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:42.858442 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:42.858442 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:42.858442 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:42.858442 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:35:42.858442 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:35:42.858442 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:35:42.858442 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:35:42.860312 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:42.860312 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:42.860312 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:42.860312 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:42.860312 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:42.860312 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:42.860312 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:42.860312 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:42.860312 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:42.860312 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:42.860312 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:42.860312 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:42.860312 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:42.860312 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:42.860312 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:42.860312 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:42.860312 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:35:42.862300 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:35:42.862300 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:42.862300 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:35:42.862300 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 513/1024] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:35:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:35:42.873515 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eff2f5-2686-4eae-a893-631544f8da6b None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:42.994811 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eff2f5-2686-4eae-a893-631544f8da6b None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:43.028512 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:43.036346 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0830f6-5cde-49ef-bf0a-ad9e0ab20944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:43.036998 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0830f6-5cde-49ef-bf0a-ad9e0ab20944 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:43.037279 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0830f6-5cde-49ef-bf0a-ad9e0ab20944 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:43.037669 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0830f6-5cde-49ef-bf0a-ad9e0ab20944 None admin] PATH_INFO: `/v3/domains` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:43.038277 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0830f6-5cde-49ef-bf0a-ad9e0ab20944 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:43.038581 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0830f6-5cde-49ef-bf0a-ad9e0ab20944 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:43.039962 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0830f6-5cde-49ef-bf0a-ad9e0ab20944 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:43.053966 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 512/1025] 10.222.0.55 () {56 vars in 1109 bytes} [Wed Aug 24 08:35:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:43.059741 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fd5e95-3927-4b53-9cda-d1ba3c73f2ef None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:43.071768 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0830f6-5cde-49ef-bf0a-ad9e0ab20944 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:43.215730 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0830f6-5cde-49ef-bf0a-ad9e0ab20944 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:43.259113 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c331e4d7-3854-4247-814e-b180740bc881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:43.259639 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c331e4d7-3854-4247-814e-b180740bc881 None admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:43.259920 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c331e4d7-3854-4247-814e-b180740bc881 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:43.260192 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c331e4d7-3854-4247-814e-b180740bc881 None admin] PATH_INFO: `/v3/services` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:43.260888 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c331e4d7-3854-4247-814e-b180740bc881 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:43.261257 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fd5e95-3927-4b53-9cda-d1ba3c73f2ef None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:43.262276 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c331e4d7-3854-4247-814e-b180740bc881 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:43.322726 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98248102-f48a-42a2-bfcf-1d23110b9e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:43.323779 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98248102-f48a-42a2-bfcf-1d23110b9e9c None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:43.324271 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98248102-f48a-42a2-bfcf-1d23110b9e9c None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:43.324786 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98248102-f48a-42a2-bfcf-1d23110b9e9c None admin] PATH_INFO: `/v3/users/nova` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:43.330194 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:43.339981 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98248102-f48a-42a2-bfcf-1d23110b9e9c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:43.343522 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98248102-f48a-42a2-bfcf-1d23110b9e9c None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:43.364967 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-98248102-f48a-42a2-bfcf-1d23110b9e9c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 08:35:43.364967 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:43.364967 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:43.364967 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:43.364967 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:43.364967 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:43.364967 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:43.364967 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:43.364967 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:43.364967 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:43.364967 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:43.364967 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:43.364967 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:35:43.364967 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:35:43.364967 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:35:43.364967 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:35:43.366274 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:43.366274 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:43.366274 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:35:43.366274 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:35:43.366274 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:35:43.366274 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:35:43.366274 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:43.366274 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:43.366274 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:43.366274 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:43.366274 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:43.366274 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:43.366274 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:43.366274 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:43.366274 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:43.366274 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:43.366274 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:43.366274 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:43.367927 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:43.367927 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:43.367927 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:35:43.367927 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:35:43.367927 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:35:43.367927 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:35:43.367927 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:35:43.367927 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:35:43.367927 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 08:35:43.367927 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:35:43.367927 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 514/1026] 10.222.0.55 () {56 vars in 1093 bytes} [Wed Aug 24 08:35:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:35:43.377093 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98248102-f48a-42a2-bfcf-1d23110b9e9c None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:43.413792 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 513/1027] 10.222.0.55 () {60 vars in 1141 bytes} [Wed Aug 24 08:35:43 2022] POST /identity/v3/services => generated 242 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:35:43.526403 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98248102-f48a-42a2-bfcf-1d23110b9e9c None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:43.579182 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c28e09e-2b91-4560-b5aa-f263a449e8ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:43.579987 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c28e09e-2b91-4560-b5aa-f263a449e8ba None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:43.580407 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c28e09e-2b91-4560-b5aa-f263a449e8ba None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:43.580852 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c28e09e-2b91-4560-b5aa-f263a449e8ba None admin] PATH_INFO: `/v3/users` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:43.581770 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c28e09e-2b91-4560-b5aa-f263a449e8ba None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:43.582217 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c28e09e-2b91-4560-b5aa-f263a449e8ba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:43.583859 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c28e09e-2b91-4560-b5aa-f263a449e8ba None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:43.606741 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 515/1028] 10.222.0.55 () {56 vars in 1133 bytes} [Wed Aug 24 08:35:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:43.614512 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c331e4d7-3854-4247-814e-b180740bc881 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:43.707258 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:43.806192 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c331e4d7-3854-4247-814e-b180740bc881 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:43.868340 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8b264e-1859-4d62-952d-5367df048d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:43.869264 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8b264e-1859-4d62-952d-5367df048d1a None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:43.869539 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8b264e-1859-4d62-952d-5367df048d1a None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:43.869899 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8b264e-1859-4d62-952d-5367df048d1a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:43.996747 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8b264e-1859-4d62-952d-5367df048d1a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:43.999649 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8b264e-1859-4d62-952d-5367df048d1a None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:44.090785 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:44.092157 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e08f0cc-b820-46d8-a8e2-85e9ccbe928b None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:44.092457 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e08f0cc-b820-46d8-a8e2-85e9ccbe928b None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:44.092768 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e08f0cc-b820-46d8-a8e2-85e9ccbe928b None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:44.094472 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 516/1029] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:35:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:35:44.100505 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27c33ec-1f79-4da6-b3f4-86c3d38012d4 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:44.109314 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27c33ec-1f79-4da6-b3f4-86c3d38012d4 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:44.109314 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27c33ec-1f79-4da6-b3f4-86c3d38012d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:44.121578 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-d27c33ec-1f79-4da6-b3f4-86c3d38012d4 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:35:44.123422 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-d27c33ec-1f79-4da6-b3f4-86c3d38012d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:35:44.177668 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 514/1030] 10.222.0.55 () {58 vars in 1282 bytes} [Wed Aug 24 08:35:43 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 564 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 08:35:44.187633 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8b264e-1859-4d62-952d-5367df048d1a None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:44.191272 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:44.234293 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 517/1031] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:35:44 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 08:35:44.331256 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8b264e-1859-4d62-952d-5367df048d1a None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:44.371977 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35985b05-2800-4765-aa84-28d6acb6ed61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:44.372804 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35985b05-2800-4765-aa84-28d6acb6ed61 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:44.373216 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35985b05-2800-4765-aa84-28d6acb6ed61 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:44.373614 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35985b05-2800-4765-aa84-28d6acb6ed61 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:44.385811 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35985b05-2800-4765-aa84-28d6acb6ed61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:44.387463 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35985b05-2800-4765-aa84-28d6acb6ed61 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:44.407519 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-35985b05-2800-4765-aa84-28d6acb6ed61 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:44.407519 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:44.407519 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:44.407519 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:44.407519 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:44.407519 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:44.407519 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:44.407519 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:44.407519 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:44.407519 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:44.407519 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:44.407519 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:44.407519 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:35:44.407519 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:35:44.407519 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:35:44.407519 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:35:44.409687 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:44.409687 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:44.409687 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:44.409687 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:44.409687 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:44.409687 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:44.409687 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:44.409687 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:44.409687 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:44.409687 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:44.409687 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:44.409687 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:44.409687 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:44.409687 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:44.409687 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:44.409687 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:44.409687 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:35:44.410872 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:35:44.410872 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:44.410872 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:35:44.410872 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 515/1032] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:35:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:44.415941 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27c33ec-1f79-4da6-b3f4-86c3d38012d4 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:44.582612 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35985b05-2800-4765-aa84-28d6acb6ed61 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:44.587066 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:44.588738 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27c33ec-1f79-4da6-b3f4-86c3d38012d4 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:44.649222 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1e5ef4-a370-4243-9bce-c89b29db5ca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:44.649917 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1e5ef4-a370-4243-9bce-c89b29db5ca5 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:44.650385 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1e5ef4-a370-4243-9bce-c89b29db5ca5 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:44.650846 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1e5ef4-a370-4243-9bce-c89b29db5ca5 None admin] PATH_INFO: `/v3/domains` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:44.651643 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1e5ef4-a370-4243-9bce-c89b29db5ca5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:44.652154 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1e5ef4-a370-4243-9bce-c89b29db5ca5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:44.653983 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1e5ef4-a370-4243-9bce-c89b29db5ca5 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:44.671051 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 518/1033] 10.222.0.55 () {56 vars in 1109 bytes} [Wed Aug 24 08:35:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:44.676770 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1e5ef4-a370-4243-9bce-c89b29db5ca5 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:44.734720 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:44.735887 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35985b05-2800-4765-aa84-28d6acb6ed61 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:44.787482 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a337d6d-f2ba-414f-a8c8-8b06660cbab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:44.788883 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a337d6d-f2ba-414f-a8c8-8b06660cbab9 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:44.789340 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a337d6d-f2ba-414f-a8c8-8b06660cbab9 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:44.789778 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a337d6d-f2ba-414f-a8c8-8b06660cbab9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:44.791489 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a337d6d-f2ba-414f-a8c8-8b06660cbab9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:44.793773 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a337d6d-f2ba-414f-a8c8-8b06660cbab9 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:44.796852 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 516/1034] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:35:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:44.802631 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a337d6d-f2ba-414f-a8c8-8b06660cbab9 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:44.828366 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1e5ef4-a370-4243-9bce-c89b29db5ca5 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:44.890638 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711b1ecc-db10-4ec5-beba-42dd2a6e5230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:44.891309 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711b1ecc-db10-4ec5-beba-42dd2a6e5230 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:44.891590 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711b1ecc-db10-4ec5-beba-42dd2a6e5230 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:44.891904 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711b1ecc-db10-4ec5-beba-42dd2a6e5230 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:44.899851 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711b1ecc-db10-4ec5-beba-42dd2a6e5230 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:44.901261 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711b1ecc-db10-4ec5-beba-42dd2a6e5230 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:44.903740 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:44.914744 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-711b1ecc-db10-4ec5-beba-42dd2a6e5230 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:35:44.914744 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:44.914744 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:44.914744 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:44.914744 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:44.914744 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:44.914744 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:44.914744 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:44.914744 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:44.914744 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:44.914744 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:44.914744 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:44.914744 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:35:44.914744 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:35:44.914744 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:35:44.920958 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:35:44.920958 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:44.920958 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:44.920958 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:44.920958 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:44.920958 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:44.920958 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:44.920958 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:44.920958 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:44.920958 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:44.920958 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:44.920958 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:44.920958 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:44.920958 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:44.920958 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:44.920958 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:44.920958 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:44.921885 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:35:44.921885 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:35:44.921885 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:35:44.921885 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:35:44.921885 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:35:44.921885 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:35:44.921885 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:35:44.921885 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:35:44.921885 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 519/1035] 10.222.0.55 () {56 vars in 1111 bytes} [Wed Aug 24 08:35:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:44.925523 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711b1ecc-db10-4ec5-beba-42dd2a6e5230 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:44.976420 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a337d6d-f2ba-414f-a8c8-8b06660cbab9 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:45.030137 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9179307a-a97a-4935-b8bf-ab3a072fbdda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:45.030939 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9179307a-a97a-4935-b8bf-ab3a072fbdda None admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:45.031353 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9179307a-a97a-4935-b8bf-ab3a072fbdda None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:45.031761 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9179307a-a97a-4935-b8bf-ab3a072fbdda None admin] PATH_INFO: `/v3/users` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:45.032563 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9179307a-a97a-4935-b8bf-ab3a072fbdda None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:45.034192 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9179307a-a97a-4935-b8bf-ab3a072fbdda None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:45.080595 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:45.085415 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711b1ecc-db10-4ec5-beba-42dd2a6e5230 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:45.136569 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 517/1036] 10.222.0.55 () {60 vars in 1132 bytes} [Wed Aug 24 08:35:44 2022] POST /identity/v3/users => generated 285 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:35:45.157541 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c263c9-1472-4ada-8515-a60bbec01f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:45.158556 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c263c9-1472-4ada-8515-a60bbec01f5c None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:45.159046 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c263c9-1472-4ada-8515-a60bbec01f5c None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:45.159535 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c263c9-1472-4ada-8515-a60bbec01f5c None admin] PATH_INFO: `/v3/projects` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:45.161090 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c263c9-1472-4ada-8515-a60bbec01f5c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:45.161640 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c263c9-1472-4ada-8515-a60bbec01f5c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:45.164395 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c263c9-1472-4ada-8515-a60bbec01f5c None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:45.191603 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 520/1037] 10.222.0.55 () {56 vars in 1148 bytes} [Wed Aug 24 08:35:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:35:45.202390 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9179307a-a97a-4935-b8bf-ab3a072fbdda None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:45.397718 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9179307a-a97a-4935-b8bf-ab3a072fbdda None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:45.448768 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:45.462549 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da694fc0-d4a9-4ae4-9d8d-14088587c768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:45.463184 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da694fc0-d4a9-4ae4-9d8d-14088587c768 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:45.463458 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da694fc0-d4a9-4ae4-9d8d-14088587c768 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:45.468429 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da694fc0-d4a9-4ae4-9d8d-14088587c768 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:45.468429 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da694fc0-d4a9-4ae4-9d8d-14088587c768 None admin] RBAC: Adding query filter params (role.id=371e1f8b73f144aea9bb7d20d253e5a0, scope.project.id=420b18725f2342159646fb86438318fd, user.id=f17da357e555420799ee681bffbcc268) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:45.468429 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da694fc0-d4a9-4ae4-9d8d-14088587c768 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:45.468429 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da694fc0-d4a9-4ae4-9d8d-14088587c768 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:45.475853 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 518/1038] 10.222.0.55 () {56 vars in 1374 bytes} [Wed Aug 24 08:35:45 2022] GET /identity/v3/role_assignments?user.id=f17da357e555420799ee681bffbcc268&scope.project.id=420b18725f2342159646fb86438318fd&role.id=371e1f8b73f144aea9bb7d20d253e5a0 => generated 259 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:45.545778 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592720f0-e61d-4a0d-8cea-b9539b2e53d8 None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:45.546525 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592720f0-e61d-4a0d-8cea-b9539b2e53d8 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:45.546525 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592720f0-e61d-4a0d-8cea-b9539b2e53d8 None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:45.548110 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 521/1039] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:35:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:35:45.554625 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f7537d-05e2-477c-818e-963c22a79f59 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:45.554935 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f7537d-05e2-477c-818e-963c22a79f59 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:45.555247 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f7537d-05e2-477c-818e-963c22a79f59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:45.576985 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-a5f7537d-05e2-477c-818e-963c22a79f59 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:35:45.578262 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-a5f7537d-05e2-477c-818e-963c22a79f59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:35:45.664326 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 519/1040] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:35:45 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:35:45.934016 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592720f0-e61d-4a0d-8cea-b9539b2e53d8 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:46.120013 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592720f0-e61d-4a0d-8cea-b9539b2e53d8 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:46.188407 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb9b35a-9498-4603-b793-7ee1b79993b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:46.189419 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb9b35a-9498-4603-b793-7ee1b79993b6 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:46.189939 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb9b35a-9498-4603-b793-7ee1b79993b6 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:46.190439 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb9b35a-9498-4603-b793-7ee1b79993b6 None admin] PATH_INFO: `/v3/services/image` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:46.195944 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb9b35a-9498-4603-b793-7ee1b79993b6 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:46.195944 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb9b35a-9498-4603-b793-7ee1b79993b6 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:46.213228 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-3bb9b35a-9498-4603-b793-7ee1b79993b6 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 08:35:46.213228 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:46.213228 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 08:35:46.213228 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 08:35:46.213228 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 08:35:46.213228 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 08:35:46.213228 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 08:35:46.213228 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:35:46.213228 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 08:35:46.213228 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:35:46.213228 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:35:46.213228 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:35:46.213228 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:46.213228 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:46.213228 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:46.213228 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:46.213228 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:46.215226 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:46.215226 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:46.215226 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:46.215226 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:46.215226 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:46.215226 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:46.215226 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 08:35:46.215226 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 08:35:46.215226 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 08:35:46.215226 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 08:35:46.215226 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:46.215226 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:46.215226 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:46.215226 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:46.215226 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:46.215226 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:46.218018 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:46.218018 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:46.218018 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:46.218018 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:46.218018 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:46.218018 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:46.218018 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:46.218018 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:46.218018 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:46.218018 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:46.218018 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 08:35:46.218018 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:35:46.218018 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 08:35:46.218018 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:35:46.218018 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 522/1041] 10.222.0.55 () {56 vars in 1105 bytes} [Wed Aug 24 08:35:45 2022] GET /identity/v3/services/image => generated 86 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:35:46.221679 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f7537d-05e2-477c-818e-963c22a79f59 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:46.342098 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f7537d-05e2-477c-818e-963c22a79f59 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:46.370335 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:46.387988 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dc0726-e755-411a-a038-952467d4424d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:46.388682 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dc0726-e755-411a-a038-952467d4424d None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:46.389023 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dc0726-e755-411a-a038-952467d4424d None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:46.389301 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dc0726-e755-411a-a038-952467d4424d None admin] PATH_INFO: `/v3/services` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:46.389951 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dc0726-e755-411a-a038-952467d4424d None admin] RBAC: Adding query filter params (name=image) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:46.390251 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dc0726-e755-411a-a038-952467d4424d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:46.391683 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dc0726-e755-411a-a038-952467d4424d None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:46.397630 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 520/1042] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:35:46 2022] GET /identity/v3/services?name=image => generated 122 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:46.404336 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb9b35a-9498-4603-b793-7ee1b79993b6 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:46.453250 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:46.599336 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb9b35a-9498-4603-b793-7ee1b79993b6 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:46.667029 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29a315c-85f6-4608-a0a8-affe2b60750a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:46.668058 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29a315c-85f6-4608-a0a8-affe2b60750a None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:46.668568 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29a315c-85f6-4608-a0a8-affe2b60750a None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:46.669078 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29a315c-85f6-4608-a0a8-affe2b60750a None admin] PATH_INFO: `/v3/services` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:46.670823 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29a315c-85f6-4608-a0a8-affe2b60750a None admin] RBAC: Adding query filter params (type=image) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:46.671365 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29a315c-85f6-4608-a0a8-affe2b60750a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:46.674060 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29a315c-85f6-4608-a0a8-affe2b60750a None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:46.688889 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 523/1043] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:35:46 2022] GET /identity/v3/services?type=image => generated 350 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:35:46.693691 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dc0726-e755-411a-a038-952467d4424d None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:46.882107 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dc0726-e755-411a-a038-952467d4424d None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:46.948275 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1d5712-9fb0-4b88-b032-c2ffbedd6d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:46.949638 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1d5712-9fb0-4b88-b032-c2ffbedd6d23 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:46.950314 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1d5712-9fb0-4b88-b032-c2ffbedd6d23 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:46.951466 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1d5712-9fb0-4b88-b032-c2ffbedd6d23 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:46.952953 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1d5712-9fb0-4b88-b032-c2ffbedd6d23 None admin] RBAC: Adding query filter params (interface=public, service_id=fc6cec4f1f2a4eb1a5288094a39ee4ac, region_id=RegionOne) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:46.953688 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1d5712-9fb0-4b88-b032-c2ffbedd6d23 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:46.956313 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1d5712-9fb0-4b88-b032-c2ffbedd6d23 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:46.972578 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 521/1044] 10.222.0.55 () {56 vars in 1251 bytes} [Wed Aug 24 08:35:46 2022] GET /identity/v3/endpoints?service_id=fc6cec4f1f2a4eb1a5288094a39ee4ac&interface=public®ion_id=RegionOne => generated 194 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:46.975759 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29a315c-85f6-4608-a0a8-affe2b60750a None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:47.157513 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:47.180760 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29a315c-85f6-4608-a0a8-affe2b60750a None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:47.249927 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdfcbcf-31ce-401c-ae1b-23e9f6241f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:47.250973 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdfcbcf-31ce-401c-ae1b-23e9f6241f1d None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:47.251535 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdfcbcf-31ce-401c-ae1b-23e9f6241f1d None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:47.252024 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdfcbcf-31ce-401c-ae1b-23e9f6241f1d None admin] PATH_INFO: `/v3/services` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:47.253206 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdfcbcf-31ce-401c-ae1b-23e9f6241f1d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:47.255816 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdfcbcf-31ce-401c-ae1b-23e9f6241f1d None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:47.276863 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 524/1045] 10.222.0.55 () {56 vars in 1087 bytes} [Wed Aug 24 08:35:46 2022] GET /identity/v3/services => generated 1957 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:35:47.575752 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdb9790-4dfd-4b14-b2c3-52235f64f88a None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:47.576707 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdb9790-4dfd-4b14-b2c3-52235f64f88a None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:47.577057 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdb9790-4dfd-4b14-b2c3-52235f64f88a None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:47.578572 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 522/1046] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:35:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:35:47.584711 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9854a624-f431-47f5-8c4e-b1f9ea6f2cfa None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:47.585002 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9854a624-f431-47f5-8c4e-b1f9ea6f2cfa None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:47.585331 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9854a624-f431-47f5-8c4e-b1f9ea6f2cfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:47.606590 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-9854a624-f431-47f5-8c4e-b1f9ea6f2cfa None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:35:47.608418 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-9854a624-f431-47f5-8c4e-b1f9ea6f2cfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:35:47.700577 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 525/1047] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:35:47 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:35:48.087790 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6644b63a-5aa5-47f8-b57a-4606d13491c6 None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:48.088610 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6644b63a-5aa5-47f8-b57a-4606d13491c6 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:48.089018 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6644b63a-5aa5-47f8-b57a-4606d13491c6 None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:48.091299 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 523/1048] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:35:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:35:48.100462 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47aa39d-4f8b-49db-9f58-b201afd07360 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:48.100986 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47aa39d-4f8b-49db-9f58-b201afd07360 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:48.101449 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47aa39d-4f8b-49db-9f58-b201afd07360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:48.131072 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-c47aa39d-4f8b-49db-9f58-b201afd07360 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:35:48.132797 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-c47aa39d-4f8b-49db-9f58-b201afd07360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:35:48.197514 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6644b63a-5aa5-47f8-b57a-4606d13491c6 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:48.237031 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 526/1049] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:35:48 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:35:48.293580 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:48.392935 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6644b63a-5aa5-47f8-b57a-4606d13491c6 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:48.455906 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513d0414-aceb-42a4-bbfe-0881d521c0d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:48.457543 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513d0414-aceb-42a4-bbfe-0881d521c0d4 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:48.458277 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513d0414-aceb-42a4-bbfe-0881d521c0d4 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:48.458926 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513d0414-aceb-42a4-bbfe-0881d521c0d4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:48.478193 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513d0414-aceb-42a4-bbfe-0881d521c0d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:35:48.487323 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-513d0414-aceb-42a4-bbfe-0881d521c0d4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:35:48.487323 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:48.487323 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:48.487323 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:48.487323 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:48.487323 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:48.487323 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:48.487323 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:48.487323 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:48.487323 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:48.487323 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:48.487323 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:48.487323 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:35:48.487323 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:35:48.487323 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:35:48.487323 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:35:48.488372 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:35:48.488372 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:35:48.488372 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:35:48.488372 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:35:48.488372 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:48.488372 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:48.488372 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:48.488372 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:48.488372 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:48.488372 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:48.488372 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:48.488372 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:48.488372 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:48.488372 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:48.488372 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:48.488372 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:48.489297 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:48.489297 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:48.489297 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:48.489297 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:48.489297 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:35:48.489297 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:35:48.489297 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:35:48.489297 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:35:48.489297 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:35:48.489297 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:35:48.489297 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:35:48.489297 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:35:48.490588 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 524/1050] 10.222.0.55 () {56 vars in 1096 bytes} [Wed Aug 24 08:35:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:35:48.499786 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47aa39d-4f8b-49db-9f58-b201afd07360 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:48.516410 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:48.581524 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513d0414-aceb-42a4-bbfe-0881d521c0d4 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:48.663471 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47aa39d-4f8b-49db-9f58-b201afd07360 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:48.728734 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e268770-e361-493a-b61d-e8c960839d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:48.729774 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e268770-e361-493a-b61d-e8c960839d2e None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:48.730278 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e268770-e361-493a-b61d-e8c960839d2e None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:48.730771 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e268770-e361-493a-b61d-e8c960839d2e None admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:48.732212 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e268770-e361-493a-b61d-e8c960839d2e None admin] RBAC: Adding query filter params (name=admin) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:48.733266 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e268770-e361-493a-b61d-e8c960839d2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:48.736034 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e268770-e361-493a-b61d-e8c960839d2e None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:48.756600 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 527/1051] 10.222.0.55 () {56 vars in 1099 bytes} [Wed Aug 24 08:35:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:35:48.758560 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e268770-e361-493a-b61d-e8c960839d2e None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:48.776711 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513d0414-aceb-42a4-bbfe-0881d521c0d4 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:48.790157 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:48.824885 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7786c2-27ac-4c32-869d-b2765d0c60cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:48.826599 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7786c2-27ac-4c32-869d-b2765d0c60cf None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:48.827091 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7786c2-27ac-4c32-869d-b2765d0c60cf None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:48.827577 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7786c2-27ac-4c32-869d-b2765d0c60cf None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:48.846743 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7786c2-27ac-4c32-869d-b2765d0c60cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:35:48.852665 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-db7786c2-27ac-4c32-869d-b2765d0c60cf None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 08:35:48.852665 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:48.852665 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:48.852665 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:48.852665 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:48.852665 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:48.852665 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:48.852665 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:48.852665 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:48.852665 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:48.852665 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:48.852665 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:48.852665 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:35:48.852665 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:35:48.852665 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:35:48.852665 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:35:48.853653 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:35:48.853653 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:35:48.853653 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:35:48.853653 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:35:48.853653 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:48.853653 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:48.853653 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:48.853653 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:48.853653 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:48.853653 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:48.853653 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:48.853653 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:48.853653 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:48.853653 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:48.853653 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:48.853653 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:48.854555 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:48.854555 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:48.854555 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:48.854555 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:48.854555 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:35:48.854555 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:35:48.854555 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:35:48.854555 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:35:48.854555 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:35:48.854555 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:35:48.854555 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 08:35:48.854555 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:35:48.854555 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 525/1052] 10.222.0.55 () {56 vars in 1120 bytes} [Wed Aug 24 08:35:48 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:35:48.864067 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7786c2-27ac-4c32-869d-b2765d0c60cf None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:48.876420 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:48.910265 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e268770-e361-493a-b61d-e8c960839d2e None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:48.957519 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99e74c1-b609-4771-ba26-e851295bb1e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:48.957745 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99e74c1-b609-4771-ba26-e851295bb1e7 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:48.958015 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99e74c1-b609-4771-ba26-e851295bb1e7 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:48.958291 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99e74c1-b609-4771-ba26-e851295bb1e7 None admin] PATH_INFO: `/v3/users/d64df2a41aac44c2942d1dcca8dfb49b` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:48.974494 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99e74c1-b609-4771-ba26-e851295bb1e7 None admin] RBAC: Authorizing `identity:get_user(user_id=d64df2a41aac44c2942d1dcca8dfb49b)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:48.975956 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99e74c1-b609-4771-ba26-e851295bb1e7 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:48.977950 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 528/1053] 10.222.0.55 () {56 vars in 1177 bytes} [Wed Aug 24 08:35:48 2022] GET /identity/v3/users/d64df2a41aac44c2942d1dcca8dfb49b => generated 285 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:35:48.983811 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99e74c1-b609-4771-ba26-e851295bb1e7 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:49.024437 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7786c2-27ac-4c32-869d-b2765d0c60cf None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:49.080842 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e68bb5-e66c-4817-b149-c95025085301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:49.081876 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e68bb5-e66c-4817-b149-c95025085301 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:49.082370 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e68bb5-e66c-4817-b149-c95025085301 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:49.082851 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e68bb5-e66c-4817-b149-c95025085301 None admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:49.084062 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e68bb5-e66c-4817-b149-c95025085301 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:49.084650 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e68bb5-e66c-4817-b149-c95025085301 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:49.087466 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e68bb5-e66c-4817-b149-c95025085301 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:49.104619 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 526/1054] 10.222.0.55 () {56 vars in 1115 bytes} [Wed Aug 24 08:35:48 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:35:49.111220 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e68bb5-e66c-4817-b149-c95025085301 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:49.118882 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99e74c1-b609-4771-ba26-e851295bb1e7 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:49.150545 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:49.163714 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:49.170563 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc6e452-b8b3-40df-9487-ae856d94d001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:49.174948 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc6e452-b8b3-40df-9487-ae856d94d001 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:49.174948 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc6e452-b8b3-40df-9487-ae856d94d001 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:49.174948 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc6e452-b8b3-40df-9487-ae856d94d001 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:49.250943 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e68bb5-e66c-4817-b149-c95025085301 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:49.276493 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc6e452-b8b3-40df-9487-ae856d94d001 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:49.278097 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc6e452-b8b3-40df-9487-ae856d94d001 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:49.299152 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a0c3ec-3b07-42ef-8b56-5ab306b0b461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:49.299872 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a0c3ec-3b07-42ef-8b56-5ab306b0b461 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:49.300444 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a0c3ec-3b07-42ef-8b56-5ab306b0b461 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:49.300826 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a0c3ec-3b07-42ef-8b56-5ab306b0b461 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:49.316564 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a0c3ec-3b07-42ef-8b56-5ab306b0b461 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:49.318614 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a0c3ec-3b07-42ef-8b56-5ab306b0b461 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:49.335700 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-18a0c3ec-3b07-42ef-8b56-5ab306b0b461 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:49.335700 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:49.335700 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:49.335700 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:49.335700 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:49.335700 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:49.335700 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:49.335700 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:49.335700 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:49.335700 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:49.335700 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:49.335700 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:49.335700 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:35:49.335700 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:35:49.335700 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:35:49.335700 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:35:49.337463 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:49.337463 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:49.337463 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:49.337463 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:49.337463 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:49.337463 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:49.337463 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:49.337463 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:49.337463 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:49.337463 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:49.337463 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:49.337463 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:49.337463 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:49.337463 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:49.337463 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:49.337463 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:49.337463 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:35:49.339322 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:35:49.339322 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:49.339322 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:35:49.342372 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 527/1055] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:35:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:35:49.342778 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a0c3ec-3b07-42ef-8b56-5ab306b0b461 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:49.428478 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 529/1056] 10.222.0.55 () {58 vars in 1282 bytes} [Wed Aug 24 08:35:48 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 08:35:49.435645 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc6e452-b8b3-40df-9487-ae856d94d001 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:49.442432 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:49.470395 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a0c3ec-3b07-42ef-8b56-5ab306b0b461 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:49.472643 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:49.511686 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716edee2-4125-40ad-bc10-a36e646cfc19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:49.512228 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716edee2-4125-40ad-bc10-a36e646cfc19 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:49.512504 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716edee2-4125-40ad-bc10-a36e646cfc19 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:49.512805 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716edee2-4125-40ad-bc10-a36e646cfc19 None admin] PATH_INFO: `/v3/domains` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:49.513531 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716edee2-4125-40ad-bc10-a36e646cfc19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:49.513859 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716edee2-4125-40ad-bc10-a36e646cfc19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:49.515290 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716edee2-4125-40ad-bc10-a36e646cfc19 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:49.528189 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 528/1057] 10.222.0.55 () {56 vars in 1109 bytes} [Wed Aug 24 08:35:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:49.530096 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f622b95d-419a-4d69-9713-1440134f3a0a None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:49.530384 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f622b95d-419a-4d69-9713-1440134f3a0a None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:49.530647 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f622b95d-419a-4d69-9713-1440134f3a0a None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:49.532241 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 529/1058] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:35:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:35:49.537220 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f622b95d-419a-4d69-9713-1440134f3a0a None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:49.603852 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc6e452-b8b3-40df-9487-ae856d94d001 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:49.650609 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79871ebf-7e4d-4877-a3b3-ce4f0e7ec8f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:49.651261 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79871ebf-7e4d-4877-a3b3-ce4f0e7ec8f9 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:49.651600 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79871ebf-7e4d-4877-a3b3-ce4f0e7ec8f9 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:49.651805 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79871ebf-7e4d-4877-a3b3-ce4f0e7ec8f9 None admin] PATH_INFO: `/v3/projects/e617dd554fa2459889067639f65b5f82` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:49.653001 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79871ebf-7e4d-4877-a3b3-ce4f0e7ec8f9 None admin] RBAC: Authorizing `identity:get_project(project_id=e617dd554fa2459889067639f65b5f82)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:49.654983 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79871ebf-7e4d-4877-a3b3-ce4f0e7ec8f9 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:49.656217 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f622b95d-419a-4d69-9713-1440134f3a0a None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:49.656819 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 530/1059] 10.222.0.55 () {56 vars in 1186 bytes} [Wed Aug 24 08:35:49 2022] GET /identity/v3/projects/e617dd554fa2459889067639f65b5f82 => generated 311 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:35:49.657874 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c477e2a-a7ef-4eca-9475-b8793ccf39a8 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:49.658153 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c477e2a-a7ef-4eca-9475-b8793ccf39a8 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:49.658425 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c477e2a-a7ef-4eca-9475-b8793ccf39a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:49.679393 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-0c477e2a-a7ef-4eca-9475-b8793ccf39a8 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:35:49.680696 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-0c477e2a-a7ef-4eca-9475-b8793ccf39a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:35:49.702284 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f0a4dd-ac5e-4bb6-9374-9afe6e1182c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:49.703387 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f0a4dd-ac5e-4bb6-9374-9afe6e1182c3 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:49.703959 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f0a4dd-ac5e-4bb6-9374-9afe6e1182c3 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:49.704434 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f0a4dd-ac5e-4bb6-9374-9afe6e1182c3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:49.718845 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f0a4dd-ac5e-4bb6-9374-9afe6e1182c3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:49.721424 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f0a4dd-ac5e-4bb6-9374-9afe6e1182c3 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:49.740779 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-63f0a4dd-ac5e-4bb6-9374-9afe6e1182c3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 08:35:49.740779 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:49.740779 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:49.740779 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:49.740779 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:49.740779 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:49.740779 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:49.740779 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:49.740779 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:49.740779 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:49.740779 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:49.740779 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:49.740779 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:35:49.740779 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:35:49.740779 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:35:49.740779 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:35:49.741921 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:49.741921 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:49.741921 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:35:49.741921 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:35:49.741921 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:35:49.741921 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:35:49.741921 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:49.741921 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:49.741921 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:49.741921 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:49.741921 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:49.741921 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:49.741921 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:49.741921 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:49.741921 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:49.741921 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:49.741921 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:49.741921 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:49.743023 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:49.743023 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:49.743023 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:35:49.743023 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:35:49.743023 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:35:49.743023 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:35:49.743023 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:35:49.743023 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:35:49.743023 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 08:35:49.743023 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:35:49.743023 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 530/1060] 10.222.0.55 () {56 vars in 1093 bytes} [Wed Aug 24 08:35:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:49.744299 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f0a4dd-ac5e-4bb6-9374-9afe6e1182c3 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:49.746591 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:49.768852 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 531/1061] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:35:49 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:35:49.769984 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c477e2a-a7ef-4eca-9475-b8793ccf39a8 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:49.812011 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:49.897515 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f0a4dd-ac5e-4bb6-9374-9afe6e1182c3 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:49.929880 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c477e2a-a7ef-4eca-9475-b8793ccf39a8 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:49.955517 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b750d5b-7237-441f-9aba-1dcbb877c491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:49.956261 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b750d5b-7237-441f-9aba-1dcbb877c491 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:49.956573 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b750d5b-7237-441f-9aba-1dcbb877c491 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:49.956876 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b750d5b-7237-441f-9aba-1dcbb877c491 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:49.957615 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b750d5b-7237-441f-9aba-1dcbb877c491 None admin] RBAC: Adding query filter params (role.id=b870cd1acaa744e1ae1d58b9ac283821, scope.project.id=e617dd554fa2459889067639f65b5f82, user.id=d64df2a41aac44c2942d1dcca8dfb49b) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:49.957984 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b750d5b-7237-441f-9aba-1dcbb877c491 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:49.960215 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b750d5b-7237-441f-9aba-1dcbb877c491 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:49.969616 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 531/1062] 10.222.0.55 () {56 vars in 1374 bytes} [Wed Aug 24 08:35:49 2022] GET /identity/v3/role_assignments?user.id=d64df2a41aac44c2942d1dcca8dfb49b&scope.project.id=e617dd554fa2459889067639f65b5f82&role.id=b870cd1acaa744e1ae1d58b9ac283821 => generated 259 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:49.999618 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac881b7-9cc1-4c70-b4f1-d2d15c203071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:50.000486 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac881b7-9cc1-4c70-b4f1-d2d15c203071 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:50.001017 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac881b7-9cc1-4c70-b4f1-d2d15c203071 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:50.001505 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac881b7-9cc1-4c70-b4f1-d2d15c203071 None admin] PATH_INFO: `/v3/users` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:50.003029 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac881b7-9cc1-4c70-b4f1-d2d15c203071 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:50.003580 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac881b7-9cc1-4c70-b4f1-d2d15c203071 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:50.006395 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac881b7-9cc1-4c70-b4f1-d2d15c203071 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:50.043646 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 532/1063] 10.222.0.55 () {56 vars in 1133 bytes} [Wed Aug 24 08:35:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:50.052057 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b750d5b-7237-441f-9aba-1dcbb877c491 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:50.060312 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac881b7-9cc1-4c70-b4f1-d2d15c203071 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:50.171105 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:50.195997 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:50.215164 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac881b7-9cc1-4c70-b4f1-d2d15c203071 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:50.231263 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b750d5b-7237-441f-9aba-1dcbb877c491 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:50.262198 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec29fea-70e6-43f8-8048-77f9f58b6eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:50.262909 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec29fea-70e6-43f8-8048-77f9f58b6eda None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:50.263122 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec29fea-70e6-43f8-8048-77f9f58b6eda None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:50.263425 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec29fea-70e6-43f8-8048-77f9f58b6eda None admin] PATH_INFO: `/v3/services/image` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:50.264013 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec29fea-70e6-43f8-8048-77f9f58b6eda None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:50.265461 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec29fea-70e6-43f8-8048-77f9f58b6eda None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:50.275531 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-bec29fea-70e6-43f8-8048-77f9f58b6eda None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 08:35:50.275531 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:50.275531 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 08:35:50.275531 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 08:35:50.275531 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 08:35:50.275531 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 08:35:50.275531 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 08:35:50.275531 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:35:50.275531 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 08:35:50.275531 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:35:50.275531 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:35:50.275531 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:35:50.275531 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:50.275531 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:50.275531 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:50.275531 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:50.275531 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:50.277455 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:50.277455 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:50.277455 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:50.277455 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:50.277455 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:50.277455 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:50.277455 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 08:35:50.277455 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 08:35:50.277455 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 08:35:50.277455 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 08:35:50.277455 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:50.277455 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:50.277455 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:50.277455 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:50.277455 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:50.277455 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:50.279631 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:50.279631 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:50.279631 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:50.279631 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:50.279631 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:50.279631 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:50.279631 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:50.279631 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:50.279631 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:50.279631 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:50.279631 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 08:35:50.279631 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:35:50.279631 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 08:35:50.279631 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:35:50.279631 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 533/1064] 10.222.0.55 () {56 vars in 1105 bytes} [Wed Aug 24 08:35:50 2022] GET /identity/v3/services/image => generated 86 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:35:50.283424 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec29fea-70e6-43f8-8048-77f9f58b6eda None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:50.283723 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300bd2cb-011f-4811-9aa9-f27e3e5d4c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:50.284373 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300bd2cb-011f-4811-9aa9-f27e3e5d4c55 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:50.284737 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300bd2cb-011f-4811-9aa9-f27e3e5d4c55 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:50.285126 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300bd2cb-011f-4811-9aa9-f27e3e5d4c55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:50.299507 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300bd2cb-011f-4811-9aa9-f27e3e5d4c55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:50.300904 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300bd2cb-011f-4811-9aa9-f27e3e5d4c55 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:50.316631 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-300bd2cb-011f-4811-9aa9-f27e3e5d4c55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:50.316631 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:50.316631 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:50.316631 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:50.316631 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:50.316631 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:50.316631 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:50.316631 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:50.316631 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:50.316631 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:50.316631 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:50.316631 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:50.316631 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:35:50.316631 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:35:50.316631 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:35:50.316631 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:35:50.318460 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:50.318460 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:50.318460 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:50.318460 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:50.318460 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:50.318460 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:50.318460 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:50.318460 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:50.318460 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:50.318460 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:50.318460 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:50.318460 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:50.318460 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:50.318460 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:50.318460 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:50.318460 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:50.318460 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:35:50.320340 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:35:50.320340 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:50.320340 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:35:50.320340 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 532/1065] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:35:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:50.327765 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300bd2cb-011f-4811-9aa9-f27e3e5d4c55 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:50.423547 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec29fea-70e6-43f8-8048-77f9f58b6eda None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:50.494288 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283cdddf-79a0-4893-bb69-336f7ed971bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:50.494989 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283cdddf-79a0-4893-bb69-336f7ed971bf None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:50.497624 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283cdddf-79a0-4893-bb69-336f7ed971bf None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:50.497624 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283cdddf-79a0-4893-bb69-336f7ed971bf None admin] PATH_INFO: `/v3/services` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:50.497624 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283cdddf-79a0-4893-bb69-336f7ed971bf None admin] RBAC: Adding query filter params (name=image) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:50.498033 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283cdddf-79a0-4893-bb69-336f7ed971bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:50.500336 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283cdddf-79a0-4893-bb69-336f7ed971bf None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:50.501804 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300bd2cb-011f-4811-9aa9-f27e3e5d4c55 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:50.510832 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 534/1066] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:35:50 2022] GET /identity/v3/services?name=image => generated 122 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:35:50.516812 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283cdddf-79a0-4893-bb69-336f7ed971bf None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:50.532230 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:50.540174 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:50.560038 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db4f5a0-62d3-4bcc-93ba-248800a55b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:50.561071 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db4f5a0-62d3-4bcc-93ba-248800a55b36 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:50.561565 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db4f5a0-62d3-4bcc-93ba-248800a55b36 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:50.562059 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db4f5a0-62d3-4bcc-93ba-248800a55b36 None admin] PATH_INFO: `/v3/domains` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:50.563247 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db4f5a0-62d3-4bcc-93ba-248800a55b36 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:50.563768 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db4f5a0-62d3-4bcc-93ba-248800a55b36 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:50.566552 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db4f5a0-62d3-4bcc-93ba-248800a55b36 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:50.594366 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 533/1067] 10.222.0.55 () {56 vars in 1109 bytes} [Wed Aug 24 08:35:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:35:50.596148 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db4f5a0-62d3-4bcc-93ba-248800a55b36 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:50.687041 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283cdddf-79a0-4893-bb69-336f7ed971bf None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:50.737253 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db128aea-1b60-4d56-81b3-fef47638d306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:50.737866 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db128aea-1b60-4d56-81b3-fef47638d306 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:50.738239 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db128aea-1b60-4d56-81b3-fef47638d306 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:50.738524 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db128aea-1b60-4d56-81b3-fef47638d306 None admin] PATH_INFO: `/v3/services` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:50.739156 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db128aea-1b60-4d56-81b3-fef47638d306 None admin] RBAC: Adding query filter params (type=image) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:50.739458 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db128aea-1b60-4d56-81b3-fef47638d306 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:50.740880 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db128aea-1b60-4d56-81b3-fef47638d306 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:50.747185 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 535/1068] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:35:50 2022] GET /identity/v3/services?type=image => generated 350 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:35:50.754716 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db4f5a0-62d3-4bcc-93ba-248800a55b36 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:50.755007 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db128aea-1b60-4d56-81b3-fef47638d306 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:50.805690 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc85fab1-44fb-4100-acc3-4bbaaf131ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:50.806708 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc85fab1-44fb-4100-acc3-4bbaaf131ec6 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:50.807206 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc85fab1-44fb-4100-acc3-4bbaaf131ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:50.807621 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc85fab1-44fb-4100-acc3-4bbaaf131ec6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:50.816474 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc85fab1-44fb-4100-acc3-4bbaaf131ec6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:50.817886 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc85fab1-44fb-4100-acc3-4bbaaf131ec6 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:50.830657 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-fc85fab1-44fb-4100-acc3-4bbaaf131ec6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:35:50.830657 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:50.830657 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:50.830657 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:50.830657 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:50.830657 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:50.830657 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:50.830657 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:50.830657 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:50.830657 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:50.830657 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:50.830657 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:50.830657 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:35:50.830657 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:35:50.830657 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:35:50.832377 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:35:50.832377 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:50.832377 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:50.832377 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:50.832377 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:50.832377 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:50.832377 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:50.832377 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:50.832377 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:50.832377 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:50.832377 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:50.832377 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:50.832377 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:50.832377 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:50.832377 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:50.832377 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:50.832377 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:50.834059 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:35:50.834059 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:35:50.834059 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:35:50.834059 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:35:50.834059 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:35:50.834059 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:35:50.834059 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:35:50.834059 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:35:50.834059 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 534/1069] 10.222.0.55 () {56 vars in 1111 bytes} [Wed Aug 24 08:35:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:35:50.838111 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc85fab1-44fb-4100-acc3-4bbaaf131ec6 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:50.883005 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:50.888199 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:50.920788 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db128aea-1b60-4d56-81b3-fef47638d306 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:50.983854 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b1a654-4dcf-45a9-a25a-3df02f15c181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:50.984918 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b1a654-4dcf-45a9-a25a-3df02f15c181 None admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:50.985411 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b1a654-4dcf-45a9-a25a-3df02f15c181 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:50.985891 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b1a654-4dcf-45a9-a25a-3df02f15c181 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:50.987615 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b1a654-4dcf-45a9-a25a-3df02f15c181 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:50.990026 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b1a654-4dcf-45a9-a25a-3df02f15c181 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:50.993475 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc85fab1-44fb-4100-acc3-4bbaaf131ec6 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:51.056160 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc38e0c-85a2-4338-a400-18e358e2e7fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:51.057099 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc38e0c-85a2-4338-a400-18e358e2e7fe None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:51.057506 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc38e0c-85a2-4338-a400-18e358e2e7fe None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:51.057907 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc38e0c-85a2-4338-a400-18e358e2e7fe None admin] PATH_INFO: `/v3/projects` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:51.058971 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc38e0c-85a2-4338-a400-18e358e2e7fe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:51.059426 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc38e0c-85a2-4338-a400-18e358e2e7fe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:51.062260 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc38e0c-85a2-4338-a400-18e358e2e7fe None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:51.083142 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 535/1070] 10.222.0.55 () {56 vars in 1148 bytes} [Wed Aug 24 08:35:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:35:51.091511 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc38e0c-85a2-4338-a400-18e358e2e7fe None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:51.104570 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 536/1071] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 08:35:50 2022] POST /identity/v3/endpoints => generated 327 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:35:51.290130 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:51.297945 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc38e0c-85a2-4338-a400-18e358e2e7fe None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:51.370831 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc03648c-14f5-4467-b596-a74eebd8bc41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:51.371925 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc03648c-14f5-4467-b596-a74eebd8bc41 None admin] REQUEST_METHOD: `PUT` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:51.372410 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc03648c-14f5-4467-b596-a74eebd8bc41 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:51.373492 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc03648c-14f5-4467-b596-a74eebd8bc41 None admin] PATH_INFO: `/v3/projects/420b18725f2342159646fb86438318fd/users/f17da357e555420799ee681bffbcc268/roles/371e1f8b73f144aea9bb7d20d253e5a0` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:51.377451 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc03648c-14f5-4467-b596-a74eebd8bc41 None admin] RBAC: Authorizing `identity:create_grant(project_id=420b18725f2342159646fb86438318fd, user_id=f17da357e555420799ee681bffbcc268, role_id=371e1f8b73f144aea9bb7d20d253e5a0)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:51.380194 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc03648c-14f5-4467-b596-a74eebd8bc41 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:51.463440 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 536/1072] 10.222.0.55 () {58 vars in 1439 bytes} [Wed Aug 24 08:35:51 2022] PUT /identity/v3/projects/420b18725f2342159646fb86438318fd/users/f17da357e555420799ee681bffbcc268/roles/371e1f8b73f144aea9bb7d20d253e5a0 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:35:52.064264 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23944694-f273-4c0a-8a62-4492ab9cd787 None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:52.064885 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23944694-f273-4c0a-8a62-4492ab9cd787 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:52.064885 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23944694-f273-4c0a-8a62-4492ab9cd787 None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:52.066374 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 537/1073] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:35:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:35:52.071762 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab566a5-0a05-4773-96d1-97e3099d5fd4 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:52.072084 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab566a5-0a05-4773-96d1-97e3099d5fd4 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:52.072318 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab566a5-0a05-4773-96d1-97e3099d5fd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:52.093561 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-1ab566a5-0a05-4773-96d1-97e3099d5fd4 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:35:52.095427 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-1ab566a5-0a05-4773-96d1-97e3099d5fd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:35:52.183403 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 537/1074] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:35:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:35:52.451868 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23944694-f273-4c0a-8a62-4492ab9cd787 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:52.645483 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23944694-f273-4c0a-8a62-4492ab9cd787 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:52.713520 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58f04bf-b7cb-4c24-a461-c32c802476fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:52.715037 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58f04bf-b7cb-4c24-a461-c32c802476fb None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:52.715787 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58f04bf-b7cb-4c24-a461-c32c802476fb None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:52.716586 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58f04bf-b7cb-4c24-a461-c32c802476fb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:52.722205 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:52.739486 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58f04bf-b7cb-4c24-a461-c32c802476fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:35:52.749528 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-b58f04bf-b7cb-4c24-a461-c32c802476fb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:35:52.749528 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:52.749528 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:52.749528 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:52.749528 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:52.749528 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:52.749528 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:52.749528 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:52.749528 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:52.749528 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:52.749528 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:52.749528 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:52.749528 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:35:52.749528 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:35:52.749528 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:35:52.749528 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:35:52.751227 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:35:52.751227 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:35:52.751227 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:35:52.751227 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:35:52.751227 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:52.751227 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:52.751227 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:52.751227 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:52.751227 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:52.751227 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:52.751227 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:52.751227 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:52.751227 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:52.751227 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:52.751227 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:52.751227 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:52.753429 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:52.753429 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:52.753429 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:52.753429 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:52.753429 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:35:52.753429 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:35:52.753429 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:35:52.753429 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:35:52.753429 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:35:52.753429 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:35:52.753429 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:35:52.753429 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:35:52.753429 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 538/1075] 10.222.0.55 () {56 vars in 1096 bytes} [Wed Aug 24 08:35:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:35:52.757866 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab566a5-0a05-4773-96d1-97e3099d5fd4 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:52.878027 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:52.897451 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab566a5-0a05-4773-96d1-97e3099d5fd4 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:52.948204 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a52275a-ed75-4c33-bff9-5e0fc5d3ce1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:52.949271 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a52275a-ed75-4c33-bff9-5e0fc5d3ce1c None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:52.949758 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a52275a-ed75-4c33-bff9-5e0fc5d3ce1c None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:52.950228 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a52275a-ed75-4c33-bff9-5e0fc5d3ce1c None admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:52.951907 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a52275a-ed75-4c33-bff9-5e0fc5d3ce1c None admin] RBAC: Adding query filter params (name=admin) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:52.952907 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a52275a-ed75-4c33-bff9-5e0fc5d3ce1c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:52.956358 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a52275a-ed75-4c33-bff9-5e0fc5d3ce1c None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:52.976156 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 538/1076] 10.222.0.55 () {56 vars in 1099 bytes} [Wed Aug 24 08:35:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:35:52.978200 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58f04bf-b7cb-4c24-a461-c32c802476fb None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:53.171504 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58f04bf-b7cb-4c24-a461-c32c802476fb None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:53.244345 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd29c5e-043f-4639-ab87-85425cdd9eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:53.245392 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd29c5e-043f-4639-ab87-85425cdd9eeb None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:53.245857 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd29c5e-043f-4639-ab87-85425cdd9eeb None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:53.246346 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd29c5e-043f-4639-ab87-85425cdd9eeb None admin] PATH_INFO: `/v3/users/d64df2a41aac44c2942d1dcca8dfb49b` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:53.248148 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd29c5e-043f-4639-ab87-85425cdd9eeb None admin] RBAC: Authorizing `identity:get_user(user_id=d64df2a41aac44c2942d1dcca8dfb49b)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:53.250993 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd29c5e-043f-4639-ab87-85425cdd9eeb None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:53.259996 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 539/1077] 10.222.0.55 () {56 vars in 1177 bytes} [Wed Aug 24 08:35:52 2022] GET /identity/v3/users/d64df2a41aac44c2942d1dcca8dfb49b => generated 285 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:53.261339 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a52275a-ed75-4c33-bff9-5e0fc5d3ce1c None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:53.289072 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d717a092-e472-47b5-b575-4424302d8557 None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:53.289408 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d717a092-e472-47b5-b575-4424302d8557 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:53.289632 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d717a092-e472-47b5-b575-4424302d8557 None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:53.292168 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 540/1078] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:35:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:35:53.297909 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf94d2a-489c-4ed2-9aff-d8eb8bc7658b None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:53.298196 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf94d2a-489c-4ed2-9aff-d8eb8bc7658b None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:53.298481 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf94d2a-489c-4ed2-9aff-d8eb8bc7658b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:53.321439 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-caf94d2a-489c-4ed2-9aff-d8eb8bc7658b None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:35:53.323748 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-caf94d2a-489c-4ed2-9aff-d8eb8bc7658b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:35:53.397267 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a52275a-ed75-4c33-bff9-5e0fc5d3ce1c None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:53.408715 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:53.418857 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 541/1079] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:35:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 08:35:53.444876 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c879abb9-ebaf-41b1-8178-5442c49f58a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:53.445525 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c879abb9-ebaf-41b1-8178-5442c49f58a8 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:53.445803 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c879abb9-ebaf-41b1-8178-5442c49f58a8 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:53.446292 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c879abb9-ebaf-41b1-8178-5442c49f58a8 None admin] PATH_INFO: `/v3/projects/e617dd554fa2459889067639f65b5f82` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:53.447441 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c879abb9-ebaf-41b1-8178-5442c49f58a8 None admin] RBAC: Authorizing `identity:get_project(project_id=e617dd554fa2459889067639f65b5f82)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:53.449124 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c879abb9-ebaf-41b1-8178-5442c49f58a8 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:53.451649 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 539/1080] 10.222.0.55 () {56 vars in 1186 bytes} [Wed Aug 24 08:35:53 2022] GET /identity/v3/projects/e617dd554fa2459889067639f65b5f82 => generated 311 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:53.458621 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf94d2a-489c-4ed2-9aff-d8eb8bc7658b None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:53.655347 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf94d2a-489c-4ed2-9aff-d8eb8bc7658b None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:53.701927 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c879abb9-ebaf-41b1-8178-5442c49f58a8 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:53.719794 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:53.720215 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b81f59e-d9a3-4c06-b051-afe574bd1b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:53.721406 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b81f59e-d9a3-4c06-b051-afe574bd1b68 None admin] REQUEST_METHOD: `PUT` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:53.721850 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b81f59e-d9a3-4c06-b051-afe574bd1b68 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:53.722327 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b81f59e-d9a3-4c06-b051-afe574bd1b68 None admin] PATH_INFO: `/v3/projects/e617dd554fa2459889067639f65b5f82/users/d64df2a41aac44c2942d1dcca8dfb49b/roles/b870cd1acaa744e1ae1d58b9ac283821` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:53.725702 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b81f59e-d9a3-4c06-b051-afe574bd1b68 None admin] RBAC: Authorizing `identity:create_grant(project_id=e617dd554fa2459889067639f65b5f82, user_id=d64df2a41aac44c2942d1dcca8dfb49b, role_id=b870cd1acaa744e1ae1d58b9ac283821)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:53.728340 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b81f59e-d9a3-4c06-b051-afe574bd1b68 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:53.814431 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c879abb9-ebaf-41b1-8178-5442c49f58a8 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:53.834962 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 542/1081] 10.222.0.55 () {58 vars in 1439 bytes} [Wed Aug 24 08:35:53 2022] PUT /identity/v3/projects/e617dd554fa2459889067639f65b5f82/users/d64df2a41aac44c2942d1dcca8dfb49b/roles/b870cd1acaa744e1ae1d58b9ac283821 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:35:53.836695 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f7677b-229a-420b-bb2f-74700d3b0de3 None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:53.837101 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f7677b-229a-420b-bb2f-74700d3b0de3 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:53.837477 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f7677b-229a-420b-bb2f-74700d3b0de3 None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:53.839230 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 543/1082] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:35:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:35:53.849842 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d29322-4879-4cb7-b2f8-ec170fec7127 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:53.850180 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d29322-4879-4cb7-b2f8-ec170fec7127 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:53.850635 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d29322-4879-4cb7-b2f8-ec170fec7127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:53.868926 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1825b6f-b8e8-4194-b22d-754ba687658f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:53.869911 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1825b6f-b8e8-4194-b22d-754ba687658f None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:53.870433 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1825b6f-b8e8-4194-b22d-754ba687658f None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:53.870878 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1825b6f-b8e8-4194-b22d-754ba687658f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:53.881556 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-11d29322-4879-4cb7-b2f8-ec170fec7127 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:35:53.883723 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-11d29322-4879-4cb7-b2f8-ec170fec7127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:35:53.972139 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:54.005227 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1825b6f-b8e8-4194-b22d-754ba687658f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:54.007782 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1825b6f-b8e8-4194-b22d-754ba687658f None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:54.009787 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 544/1083] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:35:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:35:54.039399 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:54.209373 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 540/1084] 10.222.0.55 () {58 vars in 1282 bytes} [Wed Aug 24 08:35:53 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 08:35:54.215373 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d29322-4879-4cb7-b2f8-ec170fec7127 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:54.333666 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d29322-4879-4cb7-b2f8-ec170fec7127 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:54.338458 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1825b6f-b8e8-4194-b22d-754ba687658f None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:54.400779 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c289f4cf-2e5d-4df1-9279-62d51621ae90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:54.401420 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c289f4cf-2e5d-4df1-9279-62d51621ae90 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:54.401840 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c289f4cf-2e5d-4df1-9279-62d51621ae90 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:54.402003 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c289f4cf-2e5d-4df1-9279-62d51621ae90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:54.414310 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c289f4cf-2e5d-4df1-9279-62d51621ae90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:54.415783 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c289f4cf-2e5d-4df1-9279-62d51621ae90 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:54.439764 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-c289f4cf-2e5d-4df1-9279-62d51621ae90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:54.439764 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:54.439764 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:54.439764 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:54.439764 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:54.439764 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:54.439764 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:54.439764 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:54.439764 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:54.439764 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:54.439764 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:54.439764 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:54.439764 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:35:54.439764 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:35:54.439764 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:35:54.439764 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:35:54.440968 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:54.440968 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:54.440968 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:54.440968 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:54.440968 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:54.440968 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:54.440968 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:54.440968 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:54.440968 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:54.440968 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:54.440968 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:54.440968 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:54.440968 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:54.440968 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:54.440968 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:54.440968 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:54.440968 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:35:54.441886 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:35:54.441886 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:54.441886 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:35:54.447410 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 545/1085] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:35:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:35:54.449887 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c289f4cf-2e5d-4df1-9279-62d51621ae90 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:54.504682 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1825b6f-b8e8-4194-b22d-754ba687658f None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:54.514780 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:54.533573 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:54.560291 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949816d4-9456-49f2-ad34-b704a1ca8361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:54.561077 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949816d4-9456-49f2-ad34-b704a1ca8361 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:54.561355 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949816d4-9456-49f2-ad34-b704a1ca8361 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:54.561626 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949816d4-9456-49f2-ad34-b704a1ca8361 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:54.580143 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949816d4-9456-49f2-ad34-b704a1ca8361 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:35:54.589672 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-949816d4-9456-49f2-ad34-b704a1ca8361 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 08:35:54.589672 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:54.589672 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:54.589672 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:54.589672 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:54.589672 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:54.589672 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:54.589672 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:54.589672 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:54.589672 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:54.589672 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:54.589672 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:54.589672 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:35:54.589672 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:35:54.589672 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:35:54.589672 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:35:54.591625 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:35:54.591625 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:35:54.591625 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:35:54.591625 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:35:54.591625 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:54.591625 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:54.591625 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:54.591625 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:54.591625 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:54.591625 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:54.591625 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:54.591625 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:54.591625 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:54.591625 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:54.591625 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:54.591625 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:54.593415 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:54.593415 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:54.593415 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:54.593415 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:54.593415 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:35:54.593415 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:35:54.593415 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:35:54.593415 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:35:54.593415 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:35:54.593415 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:35:54.593415 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 08:35:54.593415 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:35:54.599408 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 541/1086] 10.222.0.55 () {56 vars in 1120 bytes} [Wed Aug 24 08:35:54 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:35:54.600881 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949816d4-9456-49f2-ad34-b704a1ca8361 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:54.631369 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c289f4cf-2e5d-4df1-9279-62d51621ae90 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:54.691357 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b09645-8235-4d0d-a0ea-1f5c3650e1e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:54.692530 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b09645-8235-4d0d-a0ea-1f5c3650e1e2 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:54.693207 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b09645-8235-4d0d-a0ea-1f5c3650e1e2 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:54.693867 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b09645-8235-4d0d-a0ea-1f5c3650e1e2 None admin] PATH_INFO: `/v3/domains` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:54.695161 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b09645-8235-4d0d-a0ea-1f5c3650e1e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:54.695846 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b09645-8235-4d0d-a0ea-1f5c3650e1e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:54.698193 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b09645-8235-4d0d-a0ea-1f5c3650e1e2 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:54.721451 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 546/1087] 10.222.0.55 () {56 vars in 1109 bytes} [Wed Aug 24 08:35:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:54.764660 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949816d4-9456-49f2-ad34-b704a1ca8361 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:54.807451 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab80d91-eaa9-40e2-b788-32420b629560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:54.808120 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab80d91-eaa9-40e2-b788-32420b629560 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:54.808401 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab80d91-eaa9-40e2-b788-32420b629560 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:54.808702 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab80d91-eaa9-40e2-b788-32420b629560 None admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:54.809396 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab80d91-eaa9-40e2-b788-32420b629560 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:54.809693 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab80d91-eaa9-40e2-b788-32420b629560 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:54.811634 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab80d91-eaa9-40e2-b788-32420b629560 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:54.822346 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 542/1088] 10.222.0.55 () {56 vars in 1115 bytes} [Wed Aug 24 08:35:54 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:54.828766 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b09645-8235-4d0d-a0ea-1f5c3650e1e2 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:55.018087 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:55.025590 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b09645-8235-4d0d-a0ea-1f5c3650e1e2 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:55.068211 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1a02af-ef5f-4c7e-bb8a-c9b3311eedf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:55.068899 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1a02af-ef5f-4c7e-bb8a-c9b3311eedf9 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:55.069180 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1a02af-ef5f-4c7e-bb8a-c9b3311eedf9 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:55.069463 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1a02af-ef5f-4c7e-bb8a-c9b3311eedf9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:55.082388 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1a02af-ef5f-4c7e-bb8a-c9b3311eedf9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:55.083863 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1a02af-ef5f-4c7e-bb8a-c9b3311eedf9 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:55.100744 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-8c1a02af-ef5f-4c7e-bb8a-c9b3311eedf9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:55.100744 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:55.100744 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:55.100744 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:55.100744 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:55.100744 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:55.100744 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:55.100744 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:55.100744 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:55.100744 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:55.100744 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:55.100744 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:55.100744 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:35:55.100744 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:35:55.100744 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:35:55.100744 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:35:55.102844 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:55.102844 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:55.102844 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:55.102844 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:55.102844 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:55.102844 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:55.102844 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:55.102844 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:55.102844 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:55.102844 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:55.102844 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:55.102844 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:55.102844 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:55.102844 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:55.102844 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:55.102844 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:55.102844 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:35:55.105263 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:35:55.105263 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:55.105263 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:35:55.105263 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 547/1089] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:35:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:55.111964 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab80d91-eaa9-40e2-b788-32420b629560 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:55.152249 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:55.228729 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab80d91-eaa9-40e2-b788-32420b629560 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:55.267587 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76c6a5c-fd8d-41f0-bf6d-2d27d49a8374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:55.268287 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76c6a5c-fd8d-41f0-bf6d-2d27d49a8374 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:55.268501 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76c6a5c-fd8d-41f0-bf6d-2d27d49a8374 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:55.268811 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76c6a5c-fd8d-41f0-bf6d-2d27d49a8374 None admin] PATH_INFO: `/v3/domains` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:55.269502 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76c6a5c-fd8d-41f0-bf6d-2d27d49a8374 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:55.269805 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76c6a5c-fd8d-41f0-bf6d-2d27d49a8374 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:55.271459 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76c6a5c-fd8d-41f0-bf6d-2d27d49a8374 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:55.285161 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 543/1090] 10.222.0.55 () {56 vars in 1109 bytes} [Wed Aug 24 08:35:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:55.291223 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1a02af-ef5f-4c7e-bb8a-c9b3311eedf9 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:55.410752 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1a02af-ef5f-4c7e-bb8a-c9b3311eedf9 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:55.450292 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59518a9-e45e-4538-96d6-2443974ae801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:55.451016 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59518a9-e45e-4538-96d6-2443974ae801 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:55.451793 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59518a9-e45e-4538-96d6-2443974ae801 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:55.452068 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59518a9-e45e-4538-96d6-2443974ae801 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:55.461092 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59518a9-e45e-4538-96d6-2443974ae801 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:55.462569 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59518a9-e45e-4538-96d6-2443974ae801 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:55.475270 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-d59518a9-e45e-4538-96d6-2443974ae801 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 08:35:55.475270 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:55.475270 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:55.475270 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:55.475270 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:55.475270 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:55.475270 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:55.475270 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:55.475270 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:55.475270 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:55.475270 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:55.475270 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:55.475270 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:35:55.475270 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:35:55.475270 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:35:55.475270 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:35:55.477393 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:55.477393 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:55.477393 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:35:55.477393 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:35:55.477393 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:35:55.477393 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:35:55.477393 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:55.477393 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:55.477393 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:55.477393 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:55.477393 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:55.477393 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:55.477393 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:55.477393 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:55.477393 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:55.477393 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:55.477393 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:55.477393 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:55.479705 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:55.479705 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:55.479705 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:35:55.479705 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:35:55.479705 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:35:55.479705 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:35:55.479705 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:35:55.479705 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:35:55.479705 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 08:35:55.479705 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:35:55.479705 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 548/1091] 10.222.0.55 () {56 vars in 1093 bytes} [Wed Aug 24 08:35:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:55.484693 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76c6a5c-fd8d-41f0-bf6d-2d27d49a8374 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:55.618658 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:55.628413 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76c6a5c-fd8d-41f0-bf6d-2d27d49a8374 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:55.679283 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5e6975-eca4-4d2a-b89b-263f2f0f6a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:55.679901 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5e6975-eca4-4d2a-b89b-263f2f0f6a77 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:55.680173 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5e6975-eca4-4d2a-b89b-263f2f0f6a77 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:55.680441 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5e6975-eca4-4d2a-b89b-263f2f0f6a77 None admin] PATH_INFO: `/v3/users` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:55.681231 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5e6975-eca4-4d2a-b89b-263f2f0f6a77 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:55.681505 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5e6975-eca4-4d2a-b89b-263f2f0f6a77 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:55.682955 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5e6975-eca4-4d2a-b89b-263f2f0f6a77 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:55.703683 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 544/1092] 10.222.0.55 () {56 vars in 1133 bytes} [Wed Aug 24 08:35:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:55.710267 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59518a9-e45e-4538-96d6-2443974ae801 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:55.731231 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:55.853140 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59518a9-e45e-4538-96d6-2443974ae801 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:55.892286 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee496ac-e9a4-4274-9491-33dbbd060592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:55.892972 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee496ac-e9a4-4274-9491-33dbbd060592 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:55.893337 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee496ac-e9a4-4274-9491-33dbbd060592 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:55.893707 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee496ac-e9a4-4274-9491-33dbbd060592 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:55.971247 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee496ac-e9a4-4274-9491-33dbbd060592 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:55.972741 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee496ac-e9a4-4274-9491-33dbbd060592 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:55.984821 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:56.106174 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 549/1093] 10.222.0.55 () {58 vars in 1282 bytes} [Wed Aug 24 08:35:55 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 08:35:56.107440 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5e6975-eca4-4d2a-b89b-263f2f0f6a77 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:56.229713 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5e6975-eca4-4d2a-b89b-263f2f0f6a77 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:56.270930 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7505d191-2302-405e-bed7-2967f4dac7a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:56.271561 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7505d191-2302-405e-bed7-2967f4dac7a2 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:56.271832 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7505d191-2302-405e-bed7-2967f4dac7a2 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:56.272100 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7505d191-2302-405e-bed7-2967f4dac7a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:56.284142 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7505d191-2302-405e-bed7-2967f4dac7a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:56.285631 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7505d191-2302-405e-bed7-2967f4dac7a2 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:56.289998 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ca6aa5-add9-4630-b17c-a71e3c5e9876 None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:56.290278 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ca6aa5-add9-4630-b17c-a71e3c5e9876 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:56.290555 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ca6aa5-add9-4630-b17c-a71e3c5e9876 None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:56.292141 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 550/1094] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:35:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:35:56.298634 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b28b76-d17c-4274-80be-5883b71e1122 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:56.299157 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b28b76-d17c-4274-80be-5883b71e1122 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:56.299671 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b28b76-d17c-4274-80be-5883b71e1122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:56.301649 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-7505d191-2302-405e-bed7-2967f4dac7a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:56.301649 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:56.301649 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:56.301649 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:56.301649 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:56.301649 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:56.301649 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:56.301649 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:56.301649 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:56.301649 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:56.301649 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:56.301649 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:56.301649 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:35:56.301649 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:35:56.301649 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:35:56.301649 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:35:56.302650 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:56.302650 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:56.302650 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:56.302650 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:56.302650 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:56.302650 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:56.302650 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:56.302650 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:56.302650 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:56.302650 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:56.302650 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:56.302650 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:56.302650 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:56.302650 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:56.302650 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:56.302650 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:56.302650 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:35:56.303811 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:35:56.303811 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:35:56.303811 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:35:56.303811 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 545/1095] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:35:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:56.308670 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7505d191-2302-405e-bed7-2967f4dac7a2 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:56.319687 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:56.332734 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-85b28b76-d17c-4274-80be-5883b71e1122 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:35:56.333987 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-85b28b76-d17c-4274-80be-5883b71e1122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:35:56.407595 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 551/1096] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:35:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 08:35:56.476713 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7505d191-2302-405e-bed7-2967f4dac7a2 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:56.537344 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e141c4-e1ff-414a-b9fe-3828d9d4cca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:56.538414 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e141c4-e1ff-414a-b9fe-3828d9d4cca0 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:56.538907 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e141c4-e1ff-414a-b9fe-3828d9d4cca0 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:56.539408 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e141c4-e1ff-414a-b9fe-3828d9d4cca0 None admin] PATH_INFO: `/v3/domains` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:56.540657 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e141c4-e1ff-414a-b9fe-3828d9d4cca0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:56.541185 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e141c4-e1ff-414a-b9fe-3828d9d4cca0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:56.543653 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e141c4-e1ff-414a-b9fe-3828d9d4cca0 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:56.571584 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 546/1097] 10.222.0.55 () {56 vars in 1109 bytes} [Wed Aug 24 08:35:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:35:56.573225 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b28b76-d17c-4274-80be-5883b71e1122 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:56.625395 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:56.690504 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e141c4-e1ff-414a-b9fe-3828d9d4cca0 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:56.700910 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b28b76-d17c-4274-80be-5883b71e1122 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:56.739376 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64199efa-42b0-4ed8-b42e-ea57b44cff9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:56.739923 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64199efa-42b0-4ed8-b42e-ea57b44cff9c None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:56.740253 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64199efa-42b0-4ed8-b42e-ea57b44cff9c None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:56.740530 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64199efa-42b0-4ed8-b42e-ea57b44cff9c None admin] PATH_INFO: `/v3/projects/service` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:56.747948 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64199efa-42b0-4ed8-b42e-ea57b44cff9c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:56.749426 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64199efa-42b0-4ed8-b42e-ea57b44cff9c None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:56.761453 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-64199efa-42b0-4ed8-b42e-ea57b44cff9c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:35:56.761453 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:56.761453 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:56.761453 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:56.761453 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:56.761453 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:56.761453 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:56.761453 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:56.761453 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:56.761453 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:56.761453 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:56.761453 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:56.761453 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:35:56.761453 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:35:56.761453 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:35:56.763234 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:35:56.763234 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:56.763234 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:56.763234 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:56.763234 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:56.763234 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:56.763234 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:56.763234 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:56.763234 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:56.763234 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:56.763234 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:56.763234 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:56.763234 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:56.763234 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:56.763234 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:56.763234 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:56.763234 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:56.765550 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:35:56.765550 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:35:56.765550 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:35:56.765550 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:35:56.765550 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:35:56.765550 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:35:56.765550 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:35:56.765550 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:35:56.765550 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 552/1098] 10.222.0.55 () {56 vars in 1111 bytes} [Wed Aug 24 08:35:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:35:56.769268 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64199efa-42b0-4ed8-b42e-ea57b44cff9c None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:56.796929 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:56.840251 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e141c4-e1ff-414a-b9fe-3828d9d4cca0 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:56.884753 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc24ad9-b758-4850-ad3d-a76ddbf4ce0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:56.885562 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc24ad9-b758-4850-ad3d-a76ddbf4ce0d None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:56.885759 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:56.885999 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc24ad9-b758-4850-ad3d-a76ddbf4ce0d None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:56.886380 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc24ad9-b758-4850-ad3d-a76ddbf4ce0d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:56.890511 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64199efa-42b0-4ed8-b42e-ea57b44cff9c None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:56.900109 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc24ad9-b758-4850-ad3d-a76ddbf4ce0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:35:56.908647 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-cdc24ad9-b758-4850-ad3d-a76ddbf4ce0d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:35:56.908647 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:35:56.908647 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:35:56.908647 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:35:56.908647 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:35:56.908647 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:35:56.908647 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:35:56.908647 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:35:56.908647 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:35:56.908647 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:35:56.908647 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:35:56.908647 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:35:56.908647 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:35:56.908647 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:35:56.908647 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:35:56.908647 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:35:56.909713 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:35:56.909713 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:35:56.909713 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:35:56.909713 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:35:56.909713 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:35:56.909713 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:35:56.909713 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:35:56.909713 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:35:56.909713 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:35:56.909713 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:35:56.909713 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:35:56.909713 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:35:56.909713 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:35:56.909713 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:35:56.909713 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:35:56.909713 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:35:56.910566 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:35:56.910566 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:35:56.910566 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:35:56.910566 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:35:56.910566 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:35:56.910566 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:35:56.910566 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:35:56.910566 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:35:56.910566 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:35:56.910566 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:35:56.910566 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:35:56.910566 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:35:56.910566 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 547/1099] 10.222.0.55 () {56 vars in 1096 bytes} [Wed Aug 24 08:35:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:35:56.916675 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc24ad9-b758-4850-ad3d-a76ddbf4ce0d None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:56.935156 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23352fb-59e7-4137-bb57-9da8db78e200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:56.935837 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23352fb-59e7-4137-bb57-9da8db78e200 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:56.936112 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23352fb-59e7-4137-bb57-9da8db78e200 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:56.936389 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23352fb-59e7-4137-bb57-9da8db78e200 None admin] PATH_INFO: `/v3/projects` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:56.937120 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23352fb-59e7-4137-bb57-9da8db78e200 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:56.937447 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23352fb-59e7-4137-bb57-9da8db78e200 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:56.938896 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23352fb-59e7-4137-bb57-9da8db78e200 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:56.951961 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 553/1100] 10.222.0.55 () {56 vars in 1148 bytes} [Wed Aug 24 08:35:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:35:56.959328 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23352fb-59e7-4137-bb57-9da8db78e200 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:57.065378 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc24ad9-b758-4850-ad3d-a76ddbf4ce0d None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:57.102194 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23352fb-59e7-4137-bb57-9da8db78e200 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:57.103316 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:57.114624 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464d59ef-d206-4095-a6dd-30b93696bc94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:57.115477 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464d59ef-d206-4095-a6dd-30b93696bc94 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:57.117642 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464d59ef-d206-4095-a6dd-30b93696bc94 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:57.117642 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464d59ef-d206-4095-a6dd-30b93696bc94 None admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:57.117642 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464d59ef-d206-4095-a6dd-30b93696bc94 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:57.117642 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464d59ef-d206-4095-a6dd-30b93696bc94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:57.119432 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464d59ef-d206-4095-a6dd-30b93696bc94 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:57.131863 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 548/1101] 10.222.0.55 () {56 vars in 1099 bytes} [Wed Aug 24 08:35:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:35:57.141095 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464d59ef-d206-4095-a6dd-30b93696bc94 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:57.152865 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614b7760-48cc-4188-82bb-25644997a37d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:57.153596 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614b7760-48cc-4188-82bb-25644997a37d None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:57.153934 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614b7760-48cc-4188-82bb-25644997a37d None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:57.154782 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614b7760-48cc-4188-82bb-25644997a37d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:57.155610 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614b7760-48cc-4188-82bb-25644997a37d None admin] RBAC: Adding query filter params (role.id=371e1f8b73f144aea9bb7d20d253e5a0, scope.project.id=420b18725f2342159646fb86438318fd, user.id=f17da357e555420799ee681bffbcc268) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:57.156083 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614b7760-48cc-4188-82bb-25644997a37d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:57.158014 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614b7760-48cc-4188-82bb-25644997a37d None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:57.165436 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 554/1102] 10.222.0.55 () {56 vars in 1374 bytes} [Wed Aug 24 08:35:56 2022] GET /identity/v3/role_assignments?user.id=f17da357e555420799ee681bffbcc268&scope.project.id=420b18725f2342159646fb86438318fd&role.id=371e1f8b73f144aea9bb7d20d253e5a0 => generated 608 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:35:57.281079 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464d59ef-d206-4095-a6dd-30b93696bc94 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:57.323915 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481c83aa-2f27-4dff-9e62-f3941bb26815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:57.324684 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481c83aa-2f27-4dff-9e62-f3941bb26815 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:57.325047 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481c83aa-2f27-4dff-9e62-f3941bb26815 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:57.325446 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481c83aa-2f27-4dff-9e62-f3941bb26815 None admin] PATH_INFO: `/v3/users/d64df2a41aac44c2942d1dcca8dfb49b` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:57.327055 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481c83aa-2f27-4dff-9e62-f3941bb26815 None admin] RBAC: Authorizing `identity:get_user(user_id=d64df2a41aac44c2942d1dcca8dfb49b)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:57.328651 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481c83aa-2f27-4dff-9e62-f3941bb26815 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:57.330479 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 549/1103] 10.222.0.55 () {56 vars in 1177 bytes} [Wed Aug 24 08:35:57 2022] GET /identity/v3/users/d64df2a41aac44c2942d1dcca8dfb49b => generated 285 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:35:57.337161 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614b7760-48cc-4188-82bb-25644997a37d None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:57.373417 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:57.483441 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614b7760-48cc-4188-82bb-25644997a37d None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:57.524735 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd375a4a-cbef-403a-bb30-c9931014035c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:57.525359 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd375a4a-cbef-403a-bb30-c9931014035c None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:57.525636 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd375a4a-cbef-403a-bb30-c9931014035c None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:57.525905 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd375a4a-cbef-403a-bb30-c9931014035c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:57.616215 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd375a4a-cbef-403a-bb30-c9931014035c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:57.617658 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd375a4a-cbef-403a-bb30-c9931014035c None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:57.646017 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:57.756465 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 555/1104] 10.222.0.55 () {58 vars in 1282 bytes} [Wed Aug 24 08:35:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 08:35:57.765916 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481c83aa-2f27-4dff-9e62-f3941bb26815 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:57.837920 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:57.905680 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481c83aa-2f27-4dff-9e62-f3941bb26815 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:57.953560 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8884e1f8-4e36-42ae-ba04-972e41752aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:57.954215 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8884e1f8-4e36-42ae-ba04-972e41752aef None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:57.954495 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8884e1f8-4e36-42ae-ba04-972e41752aef None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:57.954776 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8884e1f8-4e36-42ae-ba04-972e41752aef None admin] PATH_INFO: `/v3/projects/e617dd554fa2459889067639f65b5f82` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:57.955988 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8884e1f8-4e36-42ae-ba04-972e41752aef None admin] RBAC: Authorizing `identity:get_project(project_id=e617dd554fa2459889067639f65b5f82)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:57.957502 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8884e1f8-4e36-42ae-ba04-972e41752aef None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:57.959482 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 550/1105] 10.222.0.55 () {56 vars in 1186 bytes} [Wed Aug 24 08:35:57 2022] GET /identity/v3/projects/e617dd554fa2459889067639f65b5f82 => generated 311 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:35:57.968130 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd375a4a-cbef-403a-bb30-c9931014035c None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:35:58.105207 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd375a4a-cbef-403a-bb30-c9931014035c None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:35:58.134270 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:35:58.152441 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b99570-a0d2-4b7c-95d8-4344760e05a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:35:58.153341 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b99570-a0d2-4b7c-95d8-4344760e05a3 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:35:58.153701 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b99570-a0d2-4b7c-95d8-4344760e05a3 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:35:58.154275 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b99570-a0d2-4b7c-95d8-4344760e05a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:35:58.155109 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b99570-a0d2-4b7c-95d8-4344760e05a3 None admin] RBAC: Adding query filter params (role.id=b870cd1acaa744e1ae1d58b9ac283821, scope.project.id=e617dd554fa2459889067639f65b5f82, user.id=d64df2a41aac44c2942d1dcca8dfb49b) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:35:58.155653 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b99570-a0d2-4b7c-95d8-4344760e05a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:35:58.160818 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b99570-a0d2-4b7c-95d8-4344760e05a3 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:35:58.166574 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 556/1106] 10.222.0.55 () {56 vars in 1374 bytes} [Wed Aug 24 08:35:57 2022] GET /identity/v3/role_assignments?user.id=d64df2a41aac44c2942d1dcca8dfb49b&scope.project.id=e617dd554fa2459889067639f65b5f82&role.id=b870cd1acaa744e1ae1d58b9ac283821 => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:36:00.130760 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196a74f0-2f81-4ea7-ba25-da37942c1e63 None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:00.131338 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196a74f0-2f81-4ea7-ba25-da37942c1e63 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:00.131338 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196a74f0-2f81-4ea7-ba25-da37942c1e63 None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:00.132659 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 551/1107] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:36:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:36:00.138845 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e199394-eb4b-4023-9216-bab92268e811 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:00.139371 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e199394-eb4b-4023-9216-bab92268e811 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:00.139371 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e199394-eb4b-4023-9216-bab92268e811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:00.158906 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-5e199394-eb4b-4023-9216-bab92268e811 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:36:00.160117 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-5e199394-eb4b-4023-9216-bab92268e811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:36:00.238101 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 557/1108] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:36:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:36:00.760812 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196a74f0-2f81-4ea7-ba25-da37942c1e63 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:00.868379 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196a74f0-2f81-4ea7-ba25-da37942c1e63 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:00.895414 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:00.907166 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a103c55a-31d8-42be-b6cf-7eb5e51469dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:00.907806 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a103c55a-31d8-42be-b6cf-7eb5e51469dd None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:00.908079 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a103c55a-31d8-42be-b6cf-7eb5e51469dd None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:00.908346 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a103c55a-31d8-42be-b6cf-7eb5e51469dd None admin] PATH_INFO: `/v3/domains/default` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:00.909525 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a103c55a-31d8-42be-b6cf-7eb5e51469dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:00.910942 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a103c55a-31d8-42be-b6cf-7eb5e51469dd None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:00.912525 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 552/1109] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:36:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:00.918624 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e199394-eb4b-4023-9216-bab92268e811 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:01.047130 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e199394-eb4b-4023-9216-bab92268e811 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:01.064947 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:01.083670 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e890dc83-bf24-44a6-9731-22012f9e72cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:01.084351 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e890dc83-bf24-44a6-9731-22012f9e72cb None admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:01.084729 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e890dc83-bf24-44a6-9731-22012f9e72cb None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:01.085052 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e890dc83-bf24-44a6-9731-22012f9e72cb None admin] PATH_INFO: `/v3/users` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:01.085758 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e890dc83-bf24-44a6-9731-22012f9e72cb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:01.087392 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e890dc83-bf24-44a6-9731-22012f9e72cb None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:01.175036 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 558/1110] 10.222.0.55 () {60 vars in 1132 bytes} [Wed Aug 24 08:36:00 2022] POST /identity/v3/users => generated 286 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:36:03.168266 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259da367-9823-448b-b37b-2ad8e02247a7 None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:03.168917 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259da367-9823-448b-b37b-2ad8e02247a7 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:03.168917 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259da367-9823-448b-b37b-2ad8e02247a7 None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:03.170348 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 553/1111] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:36:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:36:03.176922 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96b3877-c2cd-47e5-82bd-2af77e029cfc None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:03.177469 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96b3877-c2cd-47e5-82bd-2af77e029cfc None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:03.177469 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96b3877-c2cd-47e5-82bd-2af77e029cfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:03.198326 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-d96b3877-c2cd-47e5-82bd-2af77e029cfc None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:36:03.199620 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-d96b3877-c2cd-47e5-82bd-2af77e029cfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:36:03.283388 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 559/1112] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:36:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 08:36:03.555525 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259da367-9823-448b-b37b-2ad8e02247a7 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:03.671168 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259da367-9823-448b-b37b-2ad8e02247a7 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:03.709090 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4088b8cf-9447-42f0-9aec-b8a8af44fbc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:03.709766 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4088b8cf-9447-42f0-9aec-b8a8af44fbc6 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:03.710006 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4088b8cf-9447-42f0-9aec-b8a8af44fbc6 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:03.710274 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4088b8cf-9447-42f0-9aec-b8a8af44fbc6 None admin] PATH_INFO: `/v3/roles/8b5ad96041894155b0138a676ead607e` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:03.711684 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4088b8cf-9447-42f0-9aec-b8a8af44fbc6 None admin] RBAC: Authorizing `identity:get_role(role_id=8b5ad96041894155b0138a676ead607e)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:03.713081 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4088b8cf-9447-42f0-9aec-b8a8af44fbc6 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:03.714413 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 554/1113] 10.222.0.55 () {56 vars in 1177 bytes} [Wed Aug 24 08:36:03 2022] GET /identity/v3/roles/8b5ad96041894155b0138a676ead607e => generated 223 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:03.721104 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96b3877-c2cd-47e5-82bd-2af77e029cfc None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:03.833326 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96b3877-c2cd-47e5-82bd-2af77e029cfc None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:03.834839 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:03.876152 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69970d8-acad-4c0c-bcb5-437380661503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:03.876833 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69970d8-acad-4c0c-bcb5-437380661503 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:03.877118 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69970d8-acad-4c0c-bcb5-437380661503 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:03.877394 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69970d8-acad-4c0c-bcb5-437380661503 None admin] PATH_INFO: `/v3/users/e2171f1bfa95443baa59443cd82ad91a` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:03.893381 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69970d8-acad-4c0c-bcb5-437380661503 None admin] RBAC: Authorizing `identity:get_user(user_id=e2171f1bfa95443baa59443cd82ad91a)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:03.894852 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69970d8-acad-4c0c-bcb5-437380661503 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:03.896578 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 560/1114] 10.222.0.55 () {56 vars in 1177 bytes} [Wed Aug 24 08:36:03 2022] GET /identity/v3/users/e2171f1bfa95443baa59443cd82ad91a => generated 286 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:03.906012 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4088b8cf-9447-42f0-9aec-b8a8af44fbc6 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:03.970237 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:04.018935 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4088b8cf-9447-42f0-9aec-b8a8af44fbc6 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:04.063334 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d03708-7465-4927-8159-555255c8abb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:04.064222 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d03708-7465-4927-8159-555255c8abb8 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:04.064709 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d03708-7465-4927-8159-555255c8abb8 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:04.065171 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d03708-7465-4927-8159-555255c8abb8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:04.140391 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d03708-7465-4927-8159-555255c8abb8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:04.141799 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d03708-7465-4927-8159-555255c8abb8 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:04.212293 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:04.261413 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 555/1115] 10.222.0.55 () {58 vars in 1282 bytes} [Wed Aug 24 08:36:03 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 08:36:04.268323 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69970d8-acad-4c0c-bcb5-437380661503 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:04.380218 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69970d8-acad-4c0c-bcb5-437380661503 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:04.417904 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aea530-2e5c-4ac0-9315-ec045edac1d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:04.418506 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aea530-2e5c-4ac0-9315-ec045edac1d1 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:04.418826 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aea530-2e5c-4ac0-9315-ec045edac1d1 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:04.419144 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aea530-2e5c-4ac0-9315-ec045edac1d1 None admin] PATH_INFO: `/v3/projects/e617dd554fa2459889067639f65b5f82` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:04.420386 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8aea530-2e5c-4ac0-9315-ec045edac1d1 None admin] RBAC: Authorizing `identity:get_project(project_id=e617dd554fa2459889067639f65b5f82)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:04.421911 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8aea530-2e5c-4ac0-9315-ec045edac1d1 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:04.423628 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 561/1116] 10.222.0.55 () {56 vars in 1186 bytes} [Wed Aug 24 08:36:04 2022] GET /identity/v3/projects/e617dd554fa2459889067639f65b5f82 => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:04.430064 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d03708-7465-4927-8159-555255c8abb8 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:04.535862 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d03708-7465-4927-8159-555255c8abb8 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:04.572490 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dd9d57-38de-40d9-82b3-f2693a15b6f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:04.573125 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dd9d57-38de-40d9-82b3-f2693a15b6f3 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:04.573398 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dd9d57-38de-40d9-82b3-f2693a15b6f3 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:04.573665 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dd9d57-38de-40d9-82b3-f2693a15b6f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:04.574397 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dd9d57-38de-40d9-82b3-f2693a15b6f3 None admin] RBAC: Adding query filter params (role.id=8b5ad96041894155b0138a676ead607e, scope.project.id=e617dd554fa2459889067639f65b5f82, user.id=e2171f1bfa95443baa59443cd82ad91a) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:04.574700 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dd9d57-38de-40d9-82b3-f2693a15b6f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:04.576174 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dd9d57-38de-40d9-82b3-f2693a15b6f3 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:04.582457 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 556/1117] 10.222.0.55 () {56 vars in 1374 bytes} [Wed Aug 24 08:36:04 2022] GET /identity/v3/role_assignments?user.id=e2171f1bfa95443baa59443cd82ad91a&scope.project.id=e617dd554fa2459889067639f65b5f82&role.id=8b5ad96041894155b0138a676ead607e => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:06.579103 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f4152f-ebd1-4c34-8bf9-ac0b4dd4213e None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:06.579693 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f4152f-ebd1-4c34-8bf9-ac0b4dd4213e None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:06.579693 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f4152f-ebd1-4c34-8bf9-ac0b4dd4213e None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:06.581158 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 562/1118] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:36:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:36:06.586959 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d975a45-6827-496f-af05-d91396b433af None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:06.587419 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d975a45-6827-496f-af05-d91396b433af None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:06.587593 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d975a45-6827-496f-af05-d91396b433af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:06.606533 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-0d975a45-6827-496f-af05-d91396b433af None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:36:06.607763 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-0d975a45-6827-496f-af05-d91396b433af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:36:06.627764 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:06.683143 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 557/1119] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:36:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:36:06.952811 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f4152f-ebd1-4c34-8bf9-ac0b4dd4213e None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:06.983581 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:07.063501 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f4152f-ebd1-4c34-8bf9-ac0b4dd4213e None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:07.100553 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecf2ce5-87da-4e72-a5cd-52a0ddf4d35a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:07.101203 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecf2ce5-87da-4e72-a5cd-52a0ddf4d35a None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:07.101484 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecf2ce5-87da-4e72-a5cd-52a0ddf4d35a None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:07.101755 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecf2ce5-87da-4e72-a5cd-52a0ddf4d35a None admin] PATH_INFO: `/v3/roles/8b5ad96041894155b0138a676ead607e` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:07.103189 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecf2ce5-87da-4e72-a5cd-52a0ddf4d35a None admin] RBAC: Authorizing `identity:get_role(role_id=8b5ad96041894155b0138a676ead607e)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:07.104583 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecf2ce5-87da-4e72-a5cd-52a0ddf4d35a None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:07.105936 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 563/1120] 10.222.0.55 () {56 vars in 1177 bytes} [Wed Aug 24 08:36:06 2022] GET /identity/v3/roles/8b5ad96041894155b0138a676ead607e => generated 223 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:07.115506 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d975a45-6827-496f-af05-d91396b433af None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:07.282169 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d975a45-6827-496f-af05-d91396b433af None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:07.340312 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc331b7a-f218-4693-a138-d11304a800c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:07.341409 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc331b7a-f218-4693-a138-d11304a800c8 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:07.341926 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc331b7a-f218-4693-a138-d11304a800c8 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:07.342394 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc331b7a-f218-4693-a138-d11304a800c8 None admin] PATH_INFO: `/v3/users/e2171f1bfa95443baa59443cd82ad91a` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:07.344330 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc331b7a-f218-4693-a138-d11304a800c8 None admin] RBAC: Authorizing `identity:get_user(user_id=e2171f1bfa95443baa59443cd82ad91a)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:07.347054 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc331b7a-f218-4693-a138-d11304a800c8 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:07.350034 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 558/1121] 10.222.0.55 () {56 vars in 1177 bytes} [Wed Aug 24 08:36:07 2022] GET /identity/v3/users/e2171f1bfa95443baa59443cd82ad91a => generated 286 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:07.356182 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecf2ce5-87da-4e72-a5cd-52a0ddf4d35a None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:07.464668 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:07.471594 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecf2ce5-87da-4e72-a5cd-52a0ddf4d35a None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:07.511728 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271833d6-c4f6-4f92-91f9-7cbc6743cf23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:07.512388 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271833d6-c4f6-4f92-91f9-7cbc6743cf23 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:07.512700 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271833d6-c4f6-4f92-91f9-7cbc6743cf23 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:07.512977 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271833d6-c4f6-4f92-91f9-7cbc6743cf23 None admin] PATH_INFO: `/v3/projects/e617dd554fa2459889067639f65b5f82` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:07.514176 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271833d6-c4f6-4f92-91f9-7cbc6743cf23 None admin] RBAC: Authorizing `identity:get_project(project_id=e617dd554fa2459889067639f65b5f82)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:07.515690 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271833d6-c4f6-4f92-91f9-7cbc6743cf23 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:07.517482 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 564/1122] 10.222.0.55 () {56 vars in 1186 bytes} [Wed Aug 24 08:36:07 2022] GET /identity/v3/projects/e617dd554fa2459889067639f65b5f82 => generated 311 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:07.524593 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc331b7a-f218-4693-a138-d11304a800c8 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:07.565743 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:07.683490 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc331b7a-f218-4693-a138-d11304a800c8 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:07.742613 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74518a7d-f1a4-46b9-9eec-5ea9bc2f4969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:07.743706 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74518a7d-f1a4-46b9-9eec-5ea9bc2f4969 None admin] REQUEST_METHOD: `PUT` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:07.744196 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74518a7d-f1a4-46b9-9eec-5ea9bc2f4969 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:07.744730 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74518a7d-f1a4-46b9-9eec-5ea9bc2f4969 None admin] PATH_INFO: `/v3/projects/e617dd554fa2459889067639f65b5f82/users/e2171f1bfa95443baa59443cd82ad91a/roles/8b5ad96041894155b0138a676ead607e` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:07.748365 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74518a7d-f1a4-46b9-9eec-5ea9bc2f4969 None admin] RBAC: Authorizing `identity:create_grant(project_id=e617dd554fa2459889067639f65b5f82, user_id=e2171f1bfa95443baa59443cd82ad91a, role_id=8b5ad96041894155b0138a676ead607e)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:07.751194 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74518a7d-f1a4-46b9-9eec-5ea9bc2f4969 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:07.823519 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 559/1123] 10.222.0.55 () {58 vars in 1439 bytes} [Wed Aug 24 08:36:07 2022] PUT /identity/v3/projects/e617dd554fa2459889067639f65b5f82/users/e2171f1bfa95443baa59443cd82ad91a/roles/8b5ad96041894155b0138a676ead607e => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:36:09.779037 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f89246-3390-4d6f-83e4-ecaa27e17d67 None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:09.780115 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f89246-3390-4d6f-83e4-ecaa27e17d67 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:09.780115 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f89246-3390-4d6f-83e4-ecaa27e17d67 None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:09.781470 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 565/1124] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:36:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:36:09.787208 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09341751-4234-4a7e-b555-8254a60446f8 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:09.787685 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09341751-4234-4a7e-b555-8254a60446f8 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:09.787827 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09341751-4234-4a7e-b555-8254a60446f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:09.806887 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-09341751-4234-4a7e-b555-8254a60446f8 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:36:09.808100 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-09341751-4234-4a7e-b555-8254a60446f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:36:09.879215 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 560/1125] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:36:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 08:36:10.147037 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f89246-3390-4d6f-83e4-ecaa27e17d67 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:10.253306 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f89246-3390-4d6f-83e4-ecaa27e17d67 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:10.299280 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59773123-be53-4a99-b5a8-59e23f4773bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:10.299925 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59773123-be53-4a99-b5a8-59e23f4773bb None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:10.300208 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59773123-be53-4a99-b5a8-59e23f4773bb None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:10.300489 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59773123-be53-4a99-b5a8-59e23f4773bb None admin] PATH_INFO: `/v3/roles/8b5ad96041894155b0138a676ead607e` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:10.301951 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59773123-be53-4a99-b5a8-59e23f4773bb None admin] RBAC: Authorizing `identity:get_role(role_id=8b5ad96041894155b0138a676ead607e)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:10.303314 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59773123-be53-4a99-b5a8-59e23f4773bb None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:10.304661 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 566/1126] 10.222.0.55 () {56 vars in 1177 bytes} [Wed Aug 24 08:36:10 2022] GET /identity/v3/roles/8b5ad96041894155b0138a676ead607e => generated 223 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:36:10.313595 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09341751-4234-4a7e-b555-8254a60446f8 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:10.333126 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:10.428693 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09341751-4234-4a7e-b555-8254a60446f8 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:10.465446 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36793fa8-61ce-4459-8ad3-44f3559e9674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:10.466101 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36793fa8-61ce-4459-8ad3-44f3559e9674 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:10.466380 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36793fa8-61ce-4459-8ad3-44f3559e9674 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:10.466652 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36793fa8-61ce-4459-8ad3-44f3559e9674 None admin] PATH_INFO: `/v3/users/e2171f1bfa95443baa59443cd82ad91a` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:10.467907 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36793fa8-61ce-4459-8ad3-44f3559e9674 None admin] RBAC: Authorizing `identity:get_user(user_id=e2171f1bfa95443baa59443cd82ad91a)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:10.469417 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36793fa8-61ce-4459-8ad3-44f3559e9674 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:10.471032 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 561/1127] 10.222.0.55 () {56 vars in 1177 bytes} [Wed Aug 24 08:36:10 2022] GET /identity/v3/users/e2171f1bfa95443baa59443cd82ad91a => generated 286 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:10.480886 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59773123-be53-4a99-b5a8-59e23f4773bb None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:10.535405 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:10.611821 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59773123-be53-4a99-b5a8-59e23f4773bb None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:10.649269 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f43e967-fdc7-43d8-a201-03c001818d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:10.649936 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f43e967-fdc7-43d8-a201-03c001818d28 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:10.650280 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f43e967-fdc7-43d8-a201-03c001818d28 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:10.650576 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f43e967-fdc7-43d8-a201-03c001818d28 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:10.725526 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f43e967-fdc7-43d8-a201-03c001818d28 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:10.726932 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f43e967-fdc7-43d8-a201-03c001818d28 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:10.766224 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:10.843602 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 567/1128] 10.222.0.55 () {58 vars in 1282 bytes} [Wed Aug 24 08:36:10 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 08:36:10.850697 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36793fa8-61ce-4459-8ad3-44f3559e9674 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:10.933481 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:10.953385 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36793fa8-61ce-4459-8ad3-44f3559e9674 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:10.989146 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c523ef5-ad79-44d6-b8e4-ae2e6a361ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:10.989689 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c523ef5-ad79-44d6-b8e4-ae2e6a361ea3 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:10.989965 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c523ef5-ad79-44d6-b8e4-ae2e6a361ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:10.990237 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c523ef5-ad79-44d6-b8e4-ae2e6a361ea3 None admin] PATH_INFO: `/v3/projects/e617dd554fa2459889067639f65b5f82` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:10.991397 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c523ef5-ad79-44d6-b8e4-ae2e6a361ea3 None admin] RBAC: Authorizing `identity:get_project(project_id=e617dd554fa2459889067639f65b5f82)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:10.992866 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c523ef5-ad79-44d6-b8e4-ae2e6a361ea3 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:10.994523 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 562/1129] 10.222.0.55 () {56 vars in 1186 bytes} [Wed Aug 24 08:36:10 2022] GET /identity/v3/projects/e617dd554fa2459889067639f65b5f82 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:11.001058 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f43e967-fdc7-43d8-a201-03c001818d28 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:11.107718 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f43e967-fdc7-43d8-a201-03c001818d28 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:11.138063 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:11.144168 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ed917d-4fd7-4974-812c-bd5b8be5a513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:11.144853 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ed917d-4fd7-4974-812c-bd5b8be5a513 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:11.145174 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ed917d-4fd7-4974-812c-bd5b8be5a513 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:11.145485 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ed917d-4fd7-4974-812c-bd5b8be5a513 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:11.146250 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ed917d-4fd7-4974-812c-bd5b8be5a513 None admin] RBAC: Adding query filter params (role.id=8b5ad96041894155b0138a676ead607e, scope.project.id=e617dd554fa2459889067639f65b5f82, user.id=e2171f1bfa95443baa59443cd82ad91a) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:11.146596 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ed917d-4fd7-4974-812c-bd5b8be5a513 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:11.148166 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ed917d-4fd7-4974-812c-bd5b8be5a513 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:11.155261 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 568/1130] 10.222.0.55 () {56 vars in 1374 bytes} [Wed Aug 24 08:36:10 2022] GET /identity/v3/role_assignments?user.id=e2171f1bfa95443baa59443cd82ad91a&scope.project.id=e617dd554fa2459889067639f65b5f82&role.id=8b5ad96041894155b0138a676ead607e => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:36:13.143077 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdd5254-e063-4168-ae1a-2daeebaf03e8 None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:13.143648 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdd5254-e063-4168-ae1a-2daeebaf03e8 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:13.143648 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdd5254-e063-4168-ae1a-2daeebaf03e8 None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:13.145144 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 563/1131] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:36:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:36:13.151354 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e871915a-6480-451b-9c59-3d21a27ad892 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:13.151911 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e871915a-6480-451b-9c59-3d21a27ad892 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:13.151911 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e871915a-6480-451b-9c59-3d21a27ad892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:13.171427 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-e871915a-6480-451b-9c59-3d21a27ad892 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:36:13.172684 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-e871915a-6480-451b-9c59-3d21a27ad892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:36:13.249268 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 569/1132] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:36:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 08:36:13.522527 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdd5254-e063-4168-ae1a-2daeebaf03e8 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:13.634958 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdd5254-e063-4168-ae1a-2daeebaf03e8 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:13.672339 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5cc852-28e1-49d1-8846-74b5c35cbe32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:13.673002 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5cc852-28e1-49d1-8846-74b5c35cbe32 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:13.673281 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5cc852-28e1-49d1-8846-74b5c35cbe32 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:13.673554 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5cc852-28e1-49d1-8846-74b5c35cbe32 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:13.674726 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5cc852-28e1-49d1-8846-74b5c35cbe32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:13.676511 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5cc852-28e1-49d1-8846-74b5c35cbe32 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:13.678160 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 564/1133] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:36:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:13.684309 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e871915a-6480-451b-9c59-3d21a27ad892 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:13.792506 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e871915a-6480-451b-9c59-3d21a27ad892 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:13.819954 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:13.830390 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69635f7-d012-47ee-ba20-69eafa84d637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:13.831034 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69635f7-d012-47ee-ba20-69eafa84d637 None admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:13.831316 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69635f7-d012-47ee-ba20-69eafa84d637 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:13.831592 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69635f7-d012-47ee-ba20-69eafa84d637 None admin] PATH_INFO: `/v3/projects` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:13.832280 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69635f7-d012-47ee-ba20-69eafa84d637 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:13.833839 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69635f7-d012-47ee-ba20-69eafa84d637 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:13.950348 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 570/1134] 10.222.0.55 () {60 vars in 1141 bytes} [Wed Aug 24 08:36:13 2022] POST /identity/v3/projects => generated 311 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:36:15.939067 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eb8b31-c080-4ada-b2f4-74a115c7b4e5 None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:15.939668 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eb8b31-c080-4ada-b2f4-74a115c7b4e5 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:15.939668 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eb8b31-c080-4ada-b2f4-74a115c7b4e5 None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:15.940950 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 565/1135] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:36:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:36:15.946309 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3099a558-5516-4862-bf12-528853da4395 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:15.946718 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3099a558-5516-4862-bf12-528853da4395 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:15.946855 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3099a558-5516-4862-bf12-528853da4395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:15.965496 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-3099a558-5516-4862-bf12-528853da4395 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:36:15.966742 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-3099a558-5516-4862-bf12-528853da4395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:36:16.038712 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 571/1136] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:36:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:36:16.313322 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85eb8b31-c080-4ada-b2f4-74a115c7b4e5 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:16.333969 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:16.420786 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85eb8b31-c080-4ada-b2f4-74a115c7b4e5 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:16.455564 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0507d9bf-6715-4bb2-905a-3275eb88a5e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:16.456115 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0507d9bf-6715-4bb2-905a-3275eb88a5e2 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:16.456395 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0507d9bf-6715-4bb2-905a-3275eb88a5e2 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:16.456695 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0507d9bf-6715-4bb2-905a-3275eb88a5e2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:16.457821 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0507d9bf-6715-4bb2-905a-3275eb88a5e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:16.459253 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0507d9bf-6715-4bb2-905a-3275eb88a5e2 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:16.460859 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 566/1137] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:36:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:16.467281 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3099a558-5516-4862-bf12-528853da4395 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:16.574558 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3099a558-5516-4862-bf12-528853da4395 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:16.585660 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:16.612745 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d44d07-25f2-43dd-933f-e1a9688eebb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:16.613389 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d44d07-25f2-43dd-933f-e1a9688eebb3 None admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:16.613670 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d44d07-25f2-43dd-933f-e1a9688eebb3 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:16.613943 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d44d07-25f2-43dd-933f-e1a9688eebb3 None admin] PATH_INFO: `/v3/users` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:16.614617 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d44d07-25f2-43dd-933f-e1a9688eebb3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:16.616065 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d44d07-25f2-43dd-933f-e1a9688eebb3 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:16.746370 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 572/1138] 10.222.0.55 () {60 vars in 1132 bytes} [Wed Aug 24 08:36:16 2022] POST /identity/v3/users => generated 286 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:36:18.713310 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0049a2-3b6a-4fe8-bf2b-6e7827cf7de1 None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:18.714065 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0049a2-3b6a-4fe8-bf2b-6e7827cf7de1 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:18.714065 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0049a2-3b6a-4fe8-bf2b-6e7827cf7de1 None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:18.715156 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 567/1139] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:36:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:36:18.721425 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c86b6e1-c95f-4dfa-b12b-612f3f9ede37 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:18.721920 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c86b6e1-c95f-4dfa-b12b-612f3f9ede37 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:18.722011 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c86b6e1-c95f-4dfa-b12b-612f3f9ede37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:18.741084 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-7c86b6e1-c95f-4dfa-b12b-612f3f9ede37 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:36:18.742315 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-7c86b6e1-c95f-4dfa-b12b-612f3f9ede37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:36:18.812725 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 573/1140] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:36:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 08:36:19.095437 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0049a2-3b6a-4fe8-bf2b-6e7827cf7de1 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:19.186396 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:19.208092 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0049a2-3b6a-4fe8-bf2b-6e7827cf7de1 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:19.244194 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601e5b0c-a2e3-4ac1-af79-1fa3653cf7ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:19.244850 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601e5b0c-a2e3-4ac1-af79-1fa3653cf7ac None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:19.245132 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601e5b0c-a2e3-4ac1-af79-1fa3653cf7ac None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:19.245407 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601e5b0c-a2e3-4ac1-af79-1fa3653cf7ac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:19.256343 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601e5b0c-a2e3-4ac1-af79-1fa3653cf7ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:36:19.261340 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-601e5b0c-a2e3-4ac1-af79-1fa3653cf7ac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:36:19.261340 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:19.261340 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:36:19.261340 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:36:19.261340 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:36:19.261340 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:36:19.261340 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:36:19.261340 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:36:19.261340 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:36:19.261340 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:36:19.261340 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:36:19.261340 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:36:19.261340 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:36:19.261340 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:36:19.261340 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:36:19.261340 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:36:19.262534 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:36:19.262534 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:36:19.262534 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:36:19.262534 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:36:19.262534 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:36:19.262534 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:36:19.262534 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:36:19.262534 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:36:19.262534 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:36:19.262534 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:36:19.262534 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:36:19.262534 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:36:19.262534 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:36:19.262534 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:36:19.262534 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:36:19.262534 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:36:19.263649 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:36:19.263649 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:36:19.263649 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:36:19.263649 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:36:19.263649 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:36:19.263649 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:36:19.263649 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:36:19.263649 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:36:19.263649 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:36:19.263649 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:36:19.263649 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:36:19.263649 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:36:19.263649 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 568/1141] 10.222.0.55 () {56 vars in 1096 bytes} [Wed Aug 24 08:36:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:36:19.268820 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c86b6e1-c95f-4dfa-b12b-612f3f9ede37 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:19.365130 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:19.371964 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c86b6e1-c95f-4dfa-b12b-612f3f9ede37 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:19.408021 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519822e4-32d8-4d87-97e2-16c8e12bfe93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:19.408651 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519822e4-32d8-4d87-97e2-16c8e12bfe93 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:19.408937 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519822e4-32d8-4d87-97e2-16c8e12bfe93 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:19.409209 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519822e4-32d8-4d87-97e2-16c8e12bfe93 None admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:19.409919 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519822e4-32d8-4d87-97e2-16c8e12bfe93 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:19.410220 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519822e4-32d8-4d87-97e2-16c8e12bfe93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:19.411648 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519822e4-32d8-4d87-97e2-16c8e12bfe93 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:19.419842 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 574/1142] 10.222.0.55 () {56 vars in 1099 bytes} [Wed Aug 24 08:36:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:19.426667 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601e5b0c-a2e3-4ac1-af79-1fa3653cf7ac None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:19.530762 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601e5b0c-a2e3-4ac1-af79-1fa3653cf7ac None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:19.567836 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b46501-4e02-42ab-9572-fe099c5a70a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:19.568474 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b46501-4e02-42ab-9572-fe099c5a70a3 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:19.568781 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b46501-4e02-42ab-9572-fe099c5a70a3 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:19.569049 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b46501-4e02-42ab-9572-fe099c5a70a3 None admin] PATH_INFO: `/v3/users/4ef49efd7ba54e29b4b383b9f47dd70c` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:19.583976 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b46501-4e02-42ab-9572-fe099c5a70a3 None admin] RBAC: Authorizing `identity:get_user(user_id=4ef49efd7ba54e29b4b383b9f47dd70c)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:19.585437 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b46501-4e02-42ab-9572-fe099c5a70a3 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:19.587140 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 569/1143] 10.222.0.55 () {56 vars in 1177 bytes} [Wed Aug 24 08:36:19 2022] GET /identity/v3/users/4ef49efd7ba54e29b4b383b9f47dd70c => generated 286 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:19.593977 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519822e4-32d8-4d87-97e2-16c8e12bfe93 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:19.698913 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519822e4-32d8-4d87-97e2-16c8e12bfe93 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:19.734237 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ec5d34-a4b7-490e-bfdc-b9255ee64516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:19.734941 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ec5d34-a4b7-490e-bfdc-b9255ee64516 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:19.735109 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ec5d34-a4b7-490e-bfdc-b9255ee64516 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:19.735382 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ec5d34-a4b7-490e-bfdc-b9255ee64516 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:19.761750 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:19.808020 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ec5d34-a4b7-490e-bfdc-b9255ee64516 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:19.809523 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ec5d34-a4b7-490e-bfdc-b9255ee64516 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:19.912644 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 575/1144] 10.222.0.55 () {58 vars in 1282 bytes} [Wed Aug 24 08:36:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 08:36:19.919030 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b46501-4e02-42ab-9572-fe099c5a70a3 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:19.922257 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:20.028286 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b46501-4e02-42ab-9572-fe099c5a70a3 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:20.065813 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f2dcb9-1f53-4295-aff8-e6b124aec329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:20.066542 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f2dcb9-1f53-4295-aff8-e6b124aec329 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:20.066854 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f2dcb9-1f53-4295-aff8-e6b124aec329 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:20.067159 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f2dcb9-1f53-4295-aff8-e6b124aec329 None admin] PATH_INFO: `/v3/projects/2e762809ddd94b42ad7e5ebf07cf771c` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:20.068392 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f2dcb9-1f53-4295-aff8-e6b124aec329 None admin] RBAC: Authorizing `identity:get_project(project_id=2e762809ddd94b42ad7e5ebf07cf771c)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:20.069894 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f2dcb9-1f53-4295-aff8-e6b124aec329 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:20.071786 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 570/1145] 10.222.0.55 () {56 vars in 1186 bytes} [Wed Aug 24 08:36:19 2022] GET /identity/v3/projects/2e762809ddd94b42ad7e5ebf07cf771c => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:20.078854 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ec5d34-a4b7-490e-bfdc-b9255ee64516 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:20.141401 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:20.186718 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ec5d34-a4b7-490e-bfdc-b9255ee64516 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:20.222919 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30623a9-79d1-4f6a-bb9f-14542c9fd5e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:20.223512 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30623a9-79d1-4f6a-bb9f-14542c9fd5e4 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:20.223789 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30623a9-79d1-4f6a-bb9f-14542c9fd5e4 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:20.224057 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30623a9-79d1-4f6a-bb9f-14542c9fd5e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:20.224875 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30623a9-79d1-4f6a-bb9f-14542c9fd5e4 None admin] RBAC: Adding query filter params (role.id=b870cd1acaa744e1ae1d58b9ac283821, scope.project.id=2e762809ddd94b42ad7e5ebf07cf771c, user.id=4ef49efd7ba54e29b4b383b9f47dd70c) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:20.225179 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30623a9-79d1-4f6a-bb9f-14542c9fd5e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:20.226652 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30623a9-79d1-4f6a-bb9f-14542c9fd5e4 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:20.232029 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 576/1146] 10.222.0.55 () {56 vars in 1374 bytes} [Wed Aug 24 08:36:20 2022] GET /identity/v3/role_assignments?user.id=4ef49efd7ba54e29b4b383b9f47dd70c&scope.project.id=2e762809ddd94b42ad7e5ebf07cf771c&role.id=b870cd1acaa744e1ae1d58b9ac283821 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:22.247058 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d320b25e-e9d6-47ab-96f0-9796f54d07a5 None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:22.247583 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d320b25e-e9d6-47ab-96f0-9796f54d07a5 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:22.247583 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d320b25e-e9d6-47ab-96f0-9796f54d07a5 None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:22.248978 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 571/1147] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:36:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:36:22.255211 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebdabaf-191a-41ce-afb6-8a1fdfa2133e None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:22.255733 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebdabaf-191a-41ce-afb6-8a1fdfa2133e None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:22.255859 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebdabaf-191a-41ce-afb6-8a1fdfa2133e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:22.276806 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-8ebdabaf-191a-41ce-afb6-8a1fdfa2133e None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:36:22.278072 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-8ebdabaf-191a-41ce-afb6-8a1fdfa2133e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:36:22.356071 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 577/1148] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:36:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 08:36:22.622142 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d320b25e-e9d6-47ab-96f0-9796f54d07a5 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:22.700552 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:22.738186 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d320b25e-e9d6-47ab-96f0-9796f54d07a5 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:22.779132 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95245727-24aa-4a94-9906-42fca06f6367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:22.779822 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95245727-24aa-4a94-9906-42fca06f6367 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:22.780097 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95245727-24aa-4a94-9906-42fca06f6367 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:22.780398 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95245727-24aa-4a94-9906-42fca06f6367 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:22.791769 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95245727-24aa-4a94-9906-42fca06f6367 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:36:22.796810 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-95245727-24aa-4a94-9906-42fca06f6367 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:36:22.796810 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:22.796810 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:36:22.796810 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:36:22.796810 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:36:22.796810 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:36:22.796810 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:36:22.796810 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:36:22.796810 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:36:22.796810 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:36:22.796810 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:36:22.796810 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:36:22.796810 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:36:22.796810 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:36:22.796810 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:36:22.796810 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:36:22.797984 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:36:22.797984 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:36:22.797984 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:36:22.797984 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:36:22.797984 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:36:22.797984 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:36:22.797984 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:36:22.797984 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:36:22.797984 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:36:22.797984 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:36:22.797984 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:36:22.797984 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:36:22.797984 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:36:22.797984 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:36:22.797984 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:36:22.797984 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:36:22.798883 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:36:22.798883 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:36:22.798883 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:36:22.798883 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:36:22.798883 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:36:22.798883 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:36:22.798883 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:36:22.798883 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:36:22.798883 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:36:22.798883 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:36:22.798883 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:36:22.798883 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:36:22.798883 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 572/1149] 10.222.0.55 () {56 vars in 1096 bytes} [Wed Aug 24 08:36:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:36:22.804806 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebdabaf-191a-41ce-afb6-8a1fdfa2133e None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:22.860618 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:22.912723 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebdabaf-191a-41ce-afb6-8a1fdfa2133e None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:22.949236 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad303ad5-bcb8-48d0-b744-54b776bd2b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:22.949854 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad303ad5-bcb8-48d0-b744-54b776bd2b06 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:22.950137 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad303ad5-bcb8-48d0-b744-54b776bd2b06 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:22.950414 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad303ad5-bcb8-48d0-b744-54b776bd2b06 None admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:22.951087 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad303ad5-bcb8-48d0-b744-54b776bd2b06 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:22.951428 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad303ad5-bcb8-48d0-b744-54b776bd2b06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:22.952827 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad303ad5-bcb8-48d0-b744-54b776bd2b06 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:22.961705 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 578/1150] 10.222.0.55 () {56 vars in 1099 bytes} [Wed Aug 24 08:36:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:22.967734 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95245727-24aa-4a94-9906-42fca06f6367 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:23.068884 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95245727-24aa-4a94-9906-42fca06f6367 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:23.097516 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:23.103372 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ce9f10-9fae-4a4b-ad20-6b221439476b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:23.104103 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ce9f10-9fae-4a4b-ad20-6b221439476b None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:23.104387 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ce9f10-9fae-4a4b-ad20-6b221439476b None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:23.104693 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ce9f10-9fae-4a4b-ad20-6b221439476b None admin] PATH_INFO: `/v3/users/4ef49efd7ba54e29b4b383b9f47dd70c` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:23.105823 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ce9f10-9fae-4a4b-ad20-6b221439476b None admin] RBAC: Authorizing `identity:get_user(user_id=4ef49efd7ba54e29b4b383b9f47dd70c)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:23.107264 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ce9f10-9fae-4a4b-ad20-6b221439476b None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:23.109467 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 573/1151] 10.222.0.55 () {56 vars in 1177 bytes} [Wed Aug 24 08:36:22 2022] GET /identity/v3/users/4ef49efd7ba54e29b4b383b9f47dd70c => generated 286 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:23.115422 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad303ad5-bcb8-48d0-b744-54b776bd2b06 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:23.230326 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad303ad5-bcb8-48d0-b744-54b776bd2b06 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:23.244901 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:23.268407 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e6217f-2888-4921-a792-6547b80a98f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:23.269189 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e6217f-2888-4921-a792-6547b80a98f1 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:23.269343 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e6217f-2888-4921-a792-6547b80a98f1 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:23.269616 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e6217f-2888-4921-a792-6547b80a98f1 None admin] PATH_INFO: `/v3/projects/2e762809ddd94b42ad7e5ebf07cf771c` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:23.270781 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e6217f-2888-4921-a792-6547b80a98f1 None admin] RBAC: Authorizing `identity:get_project(project_id=2e762809ddd94b42ad7e5ebf07cf771c)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:23.272233 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e6217f-2888-4921-a792-6547b80a98f1 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:23.273936 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 579/1152] 10.222.0.55 () {56 vars in 1186 bytes} [Wed Aug 24 08:36:23 2022] GET /identity/v3/projects/2e762809ddd94b42ad7e5ebf07cf771c => generated 311 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:23.280743 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ce9f10-9fae-4a4b-ad20-6b221439476b None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:23.397033 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ce9f10-9fae-4a4b-ad20-6b221439476b None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:23.432827 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca332d0-840c-4c9a-b20a-52a03dff9a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:23.433490 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca332d0-840c-4c9a-b20a-52a03dff9a86 None admin] REQUEST_METHOD: `PUT` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:23.433767 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca332d0-840c-4c9a-b20a-52a03dff9a86 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:23.434046 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca332d0-840c-4c9a-b20a-52a03dff9a86 None admin] PATH_INFO: `/v3/projects/2e762809ddd94b42ad7e5ebf07cf771c/users/4ef49efd7ba54e29b4b383b9f47dd70c/roles/b870cd1acaa744e1ae1d58b9ac283821` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:23.435978 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca332d0-840c-4c9a-b20a-52a03dff9a86 None admin] RBAC: Authorizing `identity:create_grant(project_id=2e762809ddd94b42ad7e5ebf07cf771c, user_id=4ef49efd7ba54e29b4b383b9f47dd70c, role_id=b870cd1acaa744e1ae1d58b9ac283821)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:23.437429 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca332d0-840c-4c9a-b20a-52a03dff9a86 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:23.558325 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 574/1153] 10.222.0.55 () {58 vars in 1439 bytes} [Wed Aug 24 08:36:23 2022] PUT /identity/v3/projects/2e762809ddd94b42ad7e5ebf07cf771c/users/4ef49efd7ba54e29b4b383b9f47dd70c/roles/b870cd1acaa744e1ae1d58b9ac283821 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:36:25.522708 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf22b5ac-4369-4e6d-b54e-321e77bc60ff None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:25.523246 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf22b5ac-4369-4e6d-b54e-321e77bc60ff None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:25.523246 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf22b5ac-4369-4e6d-b54e-321e77bc60ff None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:25.524673 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 580/1154] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:36:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:36:25.533502 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537155e2-1464-4439-b833-78eff85cd736 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:25.533982 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537155e2-1464-4439-b833-78eff85cd736 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:25.534102 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537155e2-1464-4439-b833-78eff85cd736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:25.552271 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-537155e2-1464-4439-b833-78eff85cd736 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:36:25.553530 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-537155e2-1464-4439-b833-78eff85cd736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:36:25.609619 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:25.624341 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 575/1155] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:36:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 08:36:25.891751 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf22b5ac-4369-4e6d-b54e-321e77bc60ff None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:26.008463 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf22b5ac-4369-4e6d-b54e-321e77bc60ff None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:26.047217 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1d536b-afe2-4a93-94cb-48d3e2a8fa0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:26.047878 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1d536b-afe2-4a93-94cb-48d3e2a8fa0e None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:26.048160 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1d536b-afe2-4a93-94cb-48d3e2a8fa0e None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:26.048434 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1d536b-afe2-4a93-94cb-48d3e2a8fa0e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:26.061116 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1d536b-afe2-4a93-94cb-48d3e2a8fa0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:36:26.065964 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-2e1d536b-afe2-4a93-94cb-48d3e2a8fa0e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:36:26.065964 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:26.065964 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:36:26.065964 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:36:26.065964 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:36:26.065964 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:36:26.065964 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:36:26.065964 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:36:26.065964 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:36:26.065964 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:36:26.065964 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:36:26.065964 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:36:26.065964 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:36:26.065964 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:36:26.065964 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:36:26.065964 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:36:26.067390 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:36:26.067390 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:36:26.067390 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:36:26.067390 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:36:26.067390 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:36:26.067390 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:36:26.067390 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:36:26.067390 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:36:26.067390 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:36:26.067390 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:36:26.067390 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:36:26.067390 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:36:26.067390 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:36:26.067390 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:36:26.067390 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:36:26.067390 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:36:26.068754 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:36:26.068754 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:36:26.068754 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:36:26.068754 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:36:26.068754 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:36:26.068754 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:36:26.068754 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:36:26.068754 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:36:26.068754 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:36:26.068754 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:36:26.068754 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:36:26.068754 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:36:26.068754 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 581/1156] 10.222.0.55 () {56 vars in 1096 bytes} [Wed Aug 24 08:36:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:36:26.073682 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537155e2-1464-4439-b833-78eff85cd736 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:26.190832 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537155e2-1464-4439-b833-78eff85cd736 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:26.225521 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599cdcfc-7ba5-4ba9-ace8-45ffb8128580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:26.226125 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599cdcfc-7ba5-4ba9-ace8-45ffb8128580 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:26.226400 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599cdcfc-7ba5-4ba9-ace8-45ffb8128580 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:26.226669 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599cdcfc-7ba5-4ba9-ace8-45ffb8128580 None admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:26.227328 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599cdcfc-7ba5-4ba9-ace8-45ffb8128580 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:26.227625 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599cdcfc-7ba5-4ba9-ace8-45ffb8128580 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:26.229016 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599cdcfc-7ba5-4ba9-ace8-45ffb8128580 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:26.237461 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 576/1157] 10.222.0.55 () {56 vars in 1099 bytes} [Wed Aug 24 08:36:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:26.243363 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1d536b-afe2-4a93-94cb-48d3e2a8fa0e None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:26.287157 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:26.347570 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1d536b-afe2-4a93-94cb-48d3e2a8fa0e None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:26.381547 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4605146-3b7d-48cb-96fb-096c2b086852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:26.382173 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4605146-3b7d-48cb-96fb-096c2b086852 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:26.382445 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4605146-3b7d-48cb-96fb-096c2b086852 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:26.382714 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4605146-3b7d-48cb-96fb-096c2b086852 None admin] PATH_INFO: `/v3/users/4ef49efd7ba54e29b4b383b9f47dd70c` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:26.383876 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4605146-3b7d-48cb-96fb-096c2b086852 None admin] RBAC: Authorizing `identity:get_user(user_id=4ef49efd7ba54e29b4b383b9f47dd70c)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:26.385604 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4605146-3b7d-48cb-96fb-096c2b086852 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:26.387208 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 582/1158] 10.222.0.55 () {56 vars in 1177 bytes} [Wed Aug 24 08:36:26 2022] GET /identity/v3/users/4ef49efd7ba54e29b4b383b9f47dd70c => generated 286 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:26.393836 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599cdcfc-7ba5-4ba9-ace8-45ffb8128580 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:26.443698 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:26.493590 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599cdcfc-7ba5-4ba9-ace8-45ffb8128580 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:26.531740 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2add750-52c0-41c5-be10-c7343d4af41a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:26.532415 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2add750-52c0-41c5-be10-c7343d4af41a None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:26.532745 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2add750-52c0-41c5-be10-c7343d4af41a None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:26.533025 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2add750-52c0-41c5-be10-c7343d4af41a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:26.605504 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2add750-52c0-41c5-be10-c7343d4af41a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:26.606947 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2add750-52c0-41c5-be10-c7343d4af41a None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:26.658123 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:26.711104 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 577/1159] 10.222.0.55 () {58 vars in 1282 bytes} [Wed Aug 24 08:36:26 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 08:36:26.717549 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4605146-3b7d-48cb-96fb-096c2b086852 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:26.834858 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4605146-3b7d-48cb-96fb-096c2b086852 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:26.849029 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:26.876551 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d8bcff-23b4-49e9-b36e-1ce15c06d6b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:26.877205 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d8bcff-23b4-49e9-b36e-1ce15c06d6b5 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:26.877479 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d8bcff-23b4-49e9-b36e-1ce15c06d6b5 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:26.877747 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d8bcff-23b4-49e9-b36e-1ce15c06d6b5 None admin] PATH_INFO: `/v3/projects/2e762809ddd94b42ad7e5ebf07cf771c` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:26.878874 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d8bcff-23b4-49e9-b36e-1ce15c06d6b5 None admin] RBAC: Authorizing `identity:get_project(project_id=2e762809ddd94b42ad7e5ebf07cf771c)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:26.880285 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d8bcff-23b4-49e9-b36e-1ce15c06d6b5 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:26.881938 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 583/1160] 10.222.0.55 () {56 vars in 1186 bytes} [Wed Aug 24 08:36:26 2022] GET /identity/v3/projects/2e762809ddd94b42ad7e5ebf07cf771c => generated 311 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:36:26.888665 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2add750-52c0-41c5-be10-c7343d4af41a None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:26.990902 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2add750-52c0-41c5-be10-c7343d4af41a None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:27.036314 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0902761e-d601-4e21-a8fd-ac9139d800c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:27.037255 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0902761e-d601-4e21-a8fd-ac9139d800c9 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:27.037759 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0902761e-d601-4e21-a8fd-ac9139d800c9 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:27.038242 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0902761e-d601-4e21-a8fd-ac9139d800c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:27.039638 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0902761e-d601-4e21-a8fd-ac9139d800c9 None admin] RBAC: Adding query filter params (role.id=b870cd1acaa744e1ae1d58b9ac283821, scope.project.id=2e762809ddd94b42ad7e5ebf07cf771c, user.id=4ef49efd7ba54e29b4b383b9f47dd70c) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:27.040208 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0902761e-d601-4e21-a8fd-ac9139d800c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:27.042757 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0902761e-d601-4e21-a8fd-ac9139d800c9 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:27.053587 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 578/1161] 10.222.0.55 () {56 vars in 1374 bytes} [Wed Aug 24 08:36:26 2022] GET /identity/v3/role_assignments?user.id=4ef49efd7ba54e29b4b383b9f47dd70c&scope.project.id=2e762809ddd94b42ad7e5ebf07cf771c&role.id=b870cd1acaa744e1ae1d58b9ac283821 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:36:29.001705 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370378d4-7e26-491f-aa7a-1fbe18342a73 None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:29.002274 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370378d4-7e26-491f-aa7a-1fbe18342a73 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:29.002370 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370378d4-7e26-491f-aa7a-1fbe18342a73 None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:29.003867 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 584/1162] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:36:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:36:29.009456 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e001bb3e-18c0-4a42-9268-4e5a082961ed None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:29.009956 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e001bb3e-18c0-4a42-9268-4e5a082961ed None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:29.010062 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e001bb3e-18c0-4a42-9268-4e5a082961ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:29.028804 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-e001bb3e-18c0-4a42-9268-4e5a082961ed None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:36:29.030030 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-e001bb3e-18c0-4a42-9268-4e5a082961ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:36:29.040447 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:29.106665 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 579/1163] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:36:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:36:29.390309 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370378d4-7e26-491f-aa7a-1fbe18342a73 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:29.496507 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370378d4-7e26-491f-aa7a-1fbe18342a73 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:29.537235 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b06c0d5-6873-4234-97e3-20fa8e86faac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:29.537857 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b06c0d5-6873-4234-97e3-20fa8e86faac None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:29.538133 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b06c0d5-6873-4234-97e3-20fa8e86faac None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:29.538404 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b06c0d5-6873-4234-97e3-20fa8e86faac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:29.590916 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:29.617316 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b06c0d5-6873-4234-97e3-20fa8e86faac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:29.619026 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b06c0d5-6873-4234-97e3-20fa8e86faac None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:29.721222 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 585/1164] 10.222.0.55 () {58 vars in 1282 bytes} [Wed Aug 24 08:36:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 08:36:29.727703 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e001bb3e-18c0-4a42-9268-4e5a082961ed None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:29.828169 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e001bb3e-18c0-4a42-9268-4e5a082961ed None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:29.862191 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5588dab-98db-4cc6-94fa-2a9228aa5abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:29.862815 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5588dab-98db-4cc6-94fa-2a9228aa5abb None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:29.863093 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5588dab-98db-4cc6-94fa-2a9228aa5abb None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:29.863362 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5588dab-98db-4cc6-94fa-2a9228aa5abb None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:29.870075 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5588dab-98db-4cc6-94fa-2a9228aa5abb None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:29.871509 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5588dab-98db-4cc6-94fa-2a9228aa5abb None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:29.882622 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-f5588dab-98db-4cc6-94fa-2a9228aa5abb None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 24 08:36:29.882622 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:29.882622 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 24 08:36:29.882622 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 24 08:36:29.882622 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:36:29.882622 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:36:29.882622 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:36:29.882622 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:36:29.882622 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 24 08:36:29.882622 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:36:29.882622 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:36:29.882622 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:36:29.882622 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:29.882622 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:36:29.882622 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:36:29.882622 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:36:29.882622 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:36:29.883835 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:36:29.883835 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:36:29.883835 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:36:29.883835 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:36:29.883835 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:36:29.883835 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:36:29.883835 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:36:29.883835 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:36:29.883835 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:36:29.883835 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:36:29.883835 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:36:29.883835 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:36:29.883835 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:36:29.883835 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:36:29.883835 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:36:29.883835 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:36:29.884926 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:36:29.884926 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:36:29.884926 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:36:29.884926 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:36:29.884926 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:36:29.884926 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:36:29.884926 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:36:29.884926 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:36:29.884926 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:36:29.884926 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:36:29.884926 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 24 08:36:29.884926 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:36:29.884926 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 24 08:36:29.884926 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:36:29.884926 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 580/1165] 10.222.0.55 () {56 vars in 1117 bytes} [Wed Aug 24 08:36:29 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:36:29.890822 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b06c0d5-6873-4234-97e3-20fa8e86faac None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:29.971999 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:29.996629 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b06c0d5-6873-4234-97e3-20fa8e86faac None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:30.032430 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdfd992-8af5-48cb-8faa-bdc7c2d18a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:30.033152 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdfd992-8af5-48cb-8faa-bdc7c2d18a90 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:30.033345 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdfd992-8af5-48cb-8faa-bdc7c2d18a90 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:30.033623 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdfd992-8af5-48cb-8faa-bdc7c2d18a90 None admin] PATH_INFO: `/v3/domains` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:30.034278 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdfd992-8af5-48cb-8faa-bdc7c2d18a90 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:30.034584 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdfd992-8af5-48cb-8faa-bdc7c2d18a90 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:30.035954 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdfd992-8af5-48cb-8faa-bdc7c2d18a90 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:30.043503 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 586/1166] 10.222.0.55 () {56 vars in 1115 bytes} [Wed Aug 24 08:36:29 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:36:31.955098 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81322f86-dfcf-485c-a849-07327932b05b None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:31.955670 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81322f86-dfcf-485c-a849-07327932b05b None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:31.955670 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81322f86-dfcf-485c-a849-07327932b05b None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:31.957003 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 581/1167] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:36:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:36:31.962622 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eada19-0313-4ae7-9feb-0d7f0ddb738f None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:31.963098 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eada19-0313-4ae7-9feb-0d7f0ddb738f None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:31.963190 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eada19-0313-4ae7-9feb-0d7f0ddb738f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:31.981473 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-24eada19-0313-4ae7-9feb-0d7f0ddb738f None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:36:31.983189 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-24eada19-0313-4ae7-9feb-0d7f0ddb738f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:36:32.055545 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 587/1168] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:36:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:36:32.328811 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81322f86-dfcf-485c-a849-07327932b05b None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:32.334149 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:32.438467 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81322f86-dfcf-485c-a849-07327932b05b None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:32.476529 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433042e7-d06e-4cd2-a02b-b1dcf219f88d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:32.477169 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433042e7-d06e-4cd2-a02b-b1dcf219f88d None admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:32.477452 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433042e7-d06e-4cd2-a02b-b1dcf219f88d None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:32.477726 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433042e7-d06e-4cd2-a02b-b1dcf219f88d None admin] PATH_INFO: `/v3/domains` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:32.478441 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433042e7-d06e-4cd2-a02b-b1dcf219f88d None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:32.480440 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433042e7-d06e-4cd2-a02b-b1dcf219f88d None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:32.559713 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 582/1169] 10.222.0.55 () {60 vars in 1138 bytes} [Wed Aug 24 08:36:32 2022] POST /identity/v3/domains => generated 267 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:36:34.510877 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e1e3c1-30d2-4bbd-9808-42d685af19d6 None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:34.511396 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e1e3c1-30d2-4bbd-9808-42d685af19d6 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:34.511396 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e1e3c1-30d2-4bbd-9808-42d685af19d6 None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:34.512705 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 588/1170] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:36:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:36:34.518568 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984a57c1-42b0-499e-9121-94498fd997d8 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:34.518993 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984a57c1-42b0-499e-9121-94498fd997d8 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:34.519112 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984a57c1-42b0-499e-9121-94498fd997d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:34.537462 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-984a57c1-42b0-499e-9121-94498fd997d8 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:36:34.538684 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-984a57c1-42b0-499e-9121-94498fd997d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:36:34.601166 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:34.610492 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 583/1171] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:36:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:36:34.876993 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e1e3c1-30d2-4bbd-9808-42d685af19d6 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:34.945592 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:34.976654 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e1e3c1-30d2-4bbd-9808-42d685af19d6 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:35.010912 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212047b8-4ff2-4719-afff-b52a628d92c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:35.011536 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212047b8-4ff2-4719-afff-b52a628d92c8 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:35.011813 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212047b8-4ff2-4719-afff-b52a628d92c8 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:35.012084 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212047b8-4ff2-4719-afff-b52a628d92c8 None admin] PATH_INFO: `/v3/domains/f0ead649a1fb4b1db20e0b1b692087fb` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:35.022376 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212047b8-4ff2-4719-afff-b52a628d92c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=f0ead649a1fb4b1db20e0b1b692087fb)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:35.023807 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212047b8-4ff2-4719-afff-b52a628d92c8 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:35.025429 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 589/1172] 10.222.0.55 () {56 vars in 1183 bytes} [Wed Aug 24 08:36:34 2022] GET /identity/v3/domains/f0ead649a1fb4b1db20e0b1b692087fb => generated 267 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:35.031614 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984a57c1-42b0-499e-9121-94498fd997d8 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:35.140190 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984a57c1-42b0-499e-9121-94498fd997d8 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:35.176893 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf44654f-fb72-4150-9203-7016def4a642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:35.177512 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf44654f-fb72-4150-9203-7016def4a642 None admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:35.177792 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf44654f-fb72-4150-9203-7016def4a642 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:35.178066 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf44654f-fb72-4150-9203-7016def4a642 None admin] PATH_INFO: `/v3/projects` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:35.178720 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf44654f-fb72-4150-9203-7016def4a642 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:35.180148 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf44654f-fb72-4150-9203-7016def4a642 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:35.280298 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 584/1173] 10.222.0.55 () {60 vars in 1141 bytes} [Wed Aug 24 08:36:35 2022] POST /identity/v3/projects => generated 361 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:36:37.235485 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0989667-f2f1-4720-abd5-1a00d4b87ee2 None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:37.236004 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0989667-f2f1-4720-abd5-1a00d4b87ee2 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:37.236004 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0989667-f2f1-4720-abd5-1a00d4b87ee2 None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:37.237393 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 590/1174] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:36:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:36:37.243178 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8348377f-973d-4eb4-894b-2ec3da4cf0e7 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:37.243639 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8348377f-973d-4eb4-894b-2ec3da4cf0e7 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:37.243738 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8348377f-973d-4eb4-894b-2ec3da4cf0e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:37.262865 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-8348377f-973d-4eb4-894b-2ec3da4cf0e7 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:36:37.264114 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-8348377f-973d-4eb4-894b-2ec3da4cf0e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:36:37.313208 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:37.337658 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 585/1175] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:36:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 08:36:37.611244 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0989667-f2f1-4720-abd5-1a00d4b87ee2 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:37.719344 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0989667-f2f1-4720-abd5-1a00d4b87ee2 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:37.749337 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:37.755140 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7e4505-00b0-4f20-a647-d15953ae9fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:37.755782 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7e4505-00b0-4f20-a647-d15953ae9fba None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:37.756070 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7e4505-00b0-4f20-a647-d15953ae9fba None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:37.756340 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7e4505-00b0-4f20-a647-d15953ae9fba None admin] PATH_INFO: `/v3/domains/f0ead649a1fb4b1db20e0b1b692087fb` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:37.757532 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7e4505-00b0-4f20-a647-d15953ae9fba None admin] RBAC: Authorizing `identity:get_domain(domain_id=f0ead649a1fb4b1db20e0b1b692087fb)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:37.759043 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7e4505-00b0-4f20-a647-d15953ae9fba None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:37.761055 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 591/1176] 10.222.0.55 () {56 vars in 1183 bytes} [Wed Aug 24 08:36:37 2022] GET /identity/v3/domains/f0ead649a1fb4b1db20e0b1b692087fb => generated 267 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:37.766955 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8348377f-973d-4eb4-894b-2ec3da4cf0e7 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:37.872303 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8348377f-973d-4eb4-894b-2ec3da4cf0e7 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:37.909185 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbe324c-1431-4386-8db1-46df567687b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:37.909800 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbe324c-1431-4386-8db1-46df567687b7 None admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:37.910080 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbe324c-1431-4386-8db1-46df567687b7 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:37.910351 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbe324c-1431-4386-8db1-46df567687b7 None admin] PATH_INFO: `/v3/users` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:37.911013 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbe324c-1431-4386-8db1-46df567687b7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:37.912438 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbe324c-1431-4386-8db1-46df567687b7 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:38.024283 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 586/1177] 10.222.0.55 () {60 vars in 1132 bytes} [Wed Aug 24 08:36:37 2022] POST /identity/v3/users => generated 311 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:36:39.978503 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9103a2d0-0f71-44a9-a45c-5665de797e30 None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:39.979057 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9103a2d0-0f71-44a9-a45c-5665de797e30 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:39.979057 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9103a2d0-0f71-44a9-a45c-5665de797e30 None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:39.980363 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 592/1178] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:36:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:36:39.985571 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f98950f-2659-4cd6-9021-9ab14d889519 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:39.985956 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f98950f-2659-4cd6-9021-9ab14d889519 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:39.986113 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f98950f-2659-4cd6-9021-9ab14d889519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:40.004138 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-5f98950f-2659-4cd6-9021-9ab14d889519 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:36:40.005348 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-5f98950f-2659-4cd6-9021-9ab14d889519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:36:40.030742 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:40.071681 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 587/1179] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:36:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:36:40.337338 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9103a2d0-0f71-44a9-a45c-5665de797e30 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:40.439819 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9103a2d0-0f71-44a9-a45c-5665de797e30 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:40.472731 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9bc1e5-043d-47d7-a110-30bdde0888af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:40.473355 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9bc1e5-043d-47d7-a110-30bdde0888af None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:40.473634 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9bc1e5-043d-47d7-a110-30bdde0888af None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:40.473902 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9bc1e5-043d-47d7-a110-30bdde0888af None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:40.484896 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9bc1e5-043d-47d7-a110-30bdde0888af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:36:40.489525 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-6c9bc1e5-043d-47d7-a110-30bdde0888af None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:36:40.489525 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:40.489525 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:36:40.489525 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:36:40.489525 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:36:40.489525 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:36:40.489525 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:36:40.489525 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:36:40.489525 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:36:40.489525 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:36:40.489525 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:36:40.489525 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:36:40.489525 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:36:40.489525 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:36:40.489525 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:36:40.489525 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:36:40.490780 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:36:40.490780 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:36:40.490780 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:36:40.490780 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:36:40.490780 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:36:40.490780 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:36:40.490780 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:36:40.490780 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:36:40.490780 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:36:40.490780 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:36:40.490780 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:36:40.490780 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:36:40.490780 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:36:40.490780 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:36:40.490780 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:36:40.490780 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:36:40.492009 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:36:40.492009 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:36:40.492009 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:36:40.492009 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:36:40.492009 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:36:40.492009 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:36:40.492009 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:36:40.492009 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:36:40.492009 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:36:40.492009 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:36:40.492009 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:36:40.492009 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:36:40.492009 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 593/1180] 10.222.0.55 () {56 vars in 1096 bytes} [Wed Aug 24 08:36:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:36:40.496652 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f98950f-2659-4cd6-9021-9ab14d889519 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:40.658051 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f98950f-2659-4cd6-9021-9ab14d889519 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:40.691274 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b76d6d-7086-400d-a0d2-71060d25c9b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:40.691879 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b76d6d-7086-400d-a0d2-71060d25c9b9 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:40.692155 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b76d6d-7086-400d-a0d2-71060d25c9b9 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:40.692425 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b76d6d-7086-400d-a0d2-71060d25c9b9 None admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:40.693115 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b76d6d-7086-400d-a0d2-71060d25c9b9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:40.693416 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b76d6d-7086-400d-a0d2-71060d25c9b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:40.694780 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b76d6d-7086-400d-a0d2-71060d25c9b9 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:40.702582 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 588/1181] 10.222.0.55 () {56 vars in 1099 bytes} [Wed Aug 24 08:36:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:40.708434 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9bc1e5-043d-47d7-a110-30bdde0888af None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:40.766317 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:40.807253 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9bc1e5-043d-47d7-a110-30bdde0888af None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:40.839766 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6f6b9e-5c14-40bb-843f-8108bad0304a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:40.840382 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6f6b9e-5c14-40bb-843f-8108bad0304a None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:40.840685 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6f6b9e-5c14-40bb-843f-8108bad0304a None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:40.840958 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6f6b9e-5c14-40bb-843f-8108bad0304a None admin] PATH_INFO: `/v3/users/673ac58961fb445c93eeb8f4665074c0` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:40.854344 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6f6b9e-5c14-40bb-843f-8108bad0304a None admin] RBAC: Authorizing `identity:get_user(user_id=673ac58961fb445c93eeb8f4665074c0)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:40.855768 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6f6b9e-5c14-40bb-843f-8108bad0304a None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:40.857380 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 594/1182] 10.222.0.55 () {56 vars in 1177 bytes} [Wed Aug 24 08:36:40 2022] GET /identity/v3/users/673ac58961fb445c93eeb8f4665074c0 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:40.863399 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b76d6d-7086-400d-a0d2-71060d25c9b9 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:40.888662 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:40.965653 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b76d6d-7086-400d-a0d2-71060d25c9b9 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:40.998585 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b61e068-f179-4dc1-b52b-509e282ed8f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:40.999239 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b61e068-f179-4dc1-b52b-509e282ed8f5 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:40.999454 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b61e068-f179-4dc1-b52b-509e282ed8f5 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:40.999725 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b61e068-f179-4dc1-b52b-509e282ed8f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:41.064983 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b61e068-f179-4dc1-b52b-509e282ed8f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:41.066415 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b61e068-f179-4dc1-b52b-509e282ed8f5 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:41.089393 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:41.162166 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 589/1183] 10.222.0.55 () {58 vars in 1282 bytes} [Wed Aug 24 08:36:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 08:36:41.168118 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6f6b9e-5c14-40bb-843f-8108bad0304a None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:41.266290 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6f6b9e-5c14-40bb-843f-8108bad0304a None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:41.282667 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:41.299920 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494195be-6ec2-4658-bf63-bf2a584cbab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:41.300568 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494195be-6ec2-4658-bf63-bf2a584cbab0 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:41.300847 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494195be-6ec2-4658-bf63-bf2a584cbab0 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:41.301200 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494195be-6ec2-4658-bf63-bf2a584cbab0 None admin] PATH_INFO: `/v3/projects/a28e631793774cb1a575e58d813008e9` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:41.302248 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494195be-6ec2-4658-bf63-bf2a584cbab0 None admin] RBAC: Authorizing `identity:get_project(project_id=a28e631793774cb1a575e58d813008e9)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:41.303670 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494195be-6ec2-4658-bf63-bf2a584cbab0 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:41.305321 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 595/1184] 10.222.0.55 () {56 vars in 1186 bytes} [Wed Aug 24 08:36:41 2022] GET /identity/v3/projects/a28e631793774cb1a575e58d813008e9 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:41.312039 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b61e068-f179-4dc1-b52b-509e282ed8f5 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:41.416086 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b61e068-f179-4dc1-b52b-509e282ed8f5 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:41.440017 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:41.450822 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65b74c0-7cc6-458c-9443-dfee034f171f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:41.451983 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65b74c0-7cc6-458c-9443-dfee034f171f None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:41.452388 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65b74c0-7cc6-458c-9443-dfee034f171f None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:41.452641 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65b74c0-7cc6-458c-9443-dfee034f171f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:41.453382 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65b74c0-7cc6-458c-9443-dfee034f171f None admin] RBAC: Adding query filter params (role.id=b870cd1acaa744e1ae1d58b9ac283821, scope.project.id=a28e631793774cb1a575e58d813008e9, user.id=673ac58961fb445c93eeb8f4665074c0) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:41.453690 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65b74c0-7cc6-458c-9443-dfee034f171f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:41.455122 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65b74c0-7cc6-458c-9443-dfee034f171f None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:41.460522 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 590/1185] 10.222.0.55 () {56 vars in 1374 bytes} [Wed Aug 24 08:36:41 2022] GET /identity/v3/role_assignments?user.id=673ac58961fb445c93eeb8f4665074c0&scope.project.id=a28e631793774cb1a575e58d813008e9&role.id=b870cd1acaa744e1ae1d58b9ac283821 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:43.328347 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24b0acb-91ba-4e4c-a49c-9b5728edad7f None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:43.328922 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24b0acb-91ba-4e4c-a49c-9b5728edad7f None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:43.328922 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24b0acb-91ba-4e4c-a49c-9b5728edad7f None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:43.330260 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 596/1186] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:36:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:36:43.335284 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f667ef5-2846-436f-9550-ca2f07ae6147 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:43.335784 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f667ef5-2846-436f-9550-ca2f07ae6147 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:43.335919 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f667ef5-2846-436f-9550-ca2f07ae6147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:43.353559 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-7f667ef5-2846-436f-9550-ca2f07ae6147 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:36:43.354730 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-7f667ef5-2846-436f-9550-ca2f07ae6147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:36:43.418905 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 591/1187] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:36:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:36:43.693855 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24b0acb-91ba-4e4c-a49c-9b5728edad7f None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:43.794112 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24b0acb-91ba-4e4c-a49c-9b5728edad7f None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:43.827480 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8118e397-af0d-42f8-9889-233ae0407bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:43.828098 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8118e397-af0d-42f8-9889-233ae0407bda None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:43.828371 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8118e397-af0d-42f8-9889-233ae0407bda None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:43.828668 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8118e397-af0d-42f8-9889-233ae0407bda None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:43.838419 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8118e397-af0d-42f8-9889-233ae0407bda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:36:43.843087 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-8118e397-af0d-42f8-9889-233ae0407bda None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:36:43.843087 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:43.843087 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:36:43.843087 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:36:43.843087 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:36:43.843087 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:36:43.843087 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:36:43.843087 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:36:43.843087 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:36:43.843087 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:36:43.843087 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:36:43.843087 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:36:43.843087 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:36:43.843087 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:36:43.843087 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:36:43.843087 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:36:43.844349 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:36:43.844349 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:36:43.844349 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:36:43.844349 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:36:43.844349 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:36:43.844349 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:36:43.844349 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:36:43.844349 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:36:43.844349 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:36:43.844349 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:36:43.844349 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:36:43.844349 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:36:43.844349 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:36:43.844349 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:36:43.844349 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:36:43.844349 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:36:43.845532 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:36:43.845532 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:36:43.845532 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:36:43.845532 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:36:43.845532 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:36:43.845532 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:36:43.845532 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:36:43.845532 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:36:43.845532 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:36:43.845532 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:36:43.845532 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:36:43.845532 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:36:43.845532 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 597/1188] 10.222.0.55 () {56 vars in 1096 bytes} [Wed Aug 24 08:36:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:36:43.850234 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f667ef5-2846-436f-9550-ca2f07ae6147 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:43.946087 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f667ef5-2846-436f-9550-ca2f07ae6147 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:43.962537 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:43.978385 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aa11eb-5e78-4c7e-a2d6-aab7f7cb2de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:43.978979 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aa11eb-5e78-4c7e-a2d6-aab7f7cb2de4 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:43.979254 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aa11eb-5e78-4c7e-a2d6-aab7f7cb2de4 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:43.979518 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aa11eb-5e78-4c7e-a2d6-aab7f7cb2de4 None admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:43.980170 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aa11eb-5e78-4c7e-a2d6-aab7f7cb2de4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:43.980466 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aa11eb-5e78-4c7e-a2d6-aab7f7cb2de4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:43.981833 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aa11eb-5e78-4c7e-a2d6-aab7f7cb2de4 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:43.990089 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 592/1189] 10.222.0.55 () {56 vars in 1099 bytes} [Wed Aug 24 08:36:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:43.996150 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8118e397-af0d-42f8-9889-233ae0407bda None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:44.043249 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:44.100003 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8118e397-af0d-42f8-9889-233ae0407bda None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:44.133789 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a60aaa2-0df3-4e7a-b6db-fe1eb6982a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:44.134400 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a60aaa2-0df3-4e7a-b6db-fe1eb6982a81 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:44.134673 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a60aaa2-0df3-4e7a-b6db-fe1eb6982a81 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:44.134938 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a60aaa2-0df3-4e7a-b6db-fe1eb6982a81 None admin] PATH_INFO: `/v3/users/673ac58961fb445c93eeb8f4665074c0` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:44.136303 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a60aaa2-0df3-4e7a-b6db-fe1eb6982a81 None admin] RBAC: Authorizing `identity:get_user(user_id=673ac58961fb445c93eeb8f4665074c0)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:44.137739 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a60aaa2-0df3-4e7a-b6db-fe1eb6982a81 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:44.139288 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 598/1190] 10.222.0.55 () {56 vars in 1177 bytes} [Wed Aug 24 08:36:43 2022] GET /identity/v3/users/673ac58961fb445c93eeb8f4665074c0 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:44.145554 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aa11eb-5e78-4c7e-a2d6-aab7f7cb2de4 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:44.243751 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aa11eb-5e78-4c7e-a2d6-aab7f7cb2de4 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:44.275647 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e54d59-4bc8-4010-8e85-bcc04f7e128a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:44.276260 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e54d59-4bc8-4010-8e85-bcc04f7e128a None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:44.276558 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e54d59-4bc8-4010-8e85-bcc04f7e128a None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:44.276822 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e54d59-4bc8-4010-8e85-bcc04f7e128a None admin] PATH_INFO: `/v3/projects/a28e631793774cb1a575e58d813008e9` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:44.277912 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e54d59-4bc8-4010-8e85-bcc04f7e128a None admin] RBAC: Authorizing `identity:get_project(project_id=a28e631793774cb1a575e58d813008e9)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:44.279806 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e54d59-4bc8-4010-8e85-bcc04f7e128a None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:44.281443 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 593/1191] 10.222.0.55 () {56 vars in 1186 bytes} [Wed Aug 24 08:36:44 2022] GET /identity/v3/projects/a28e631793774cb1a575e58d813008e9 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:44.287763 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a60aaa2-0df3-4e7a-b6db-fe1eb6982a81 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:44.385780 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a60aaa2-0df3-4e7a-b6db-fe1eb6982a81 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:44.398718 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:44.419423 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ff2525-88b2-478a-848d-6cbe361e5e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:44.420732 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ff2525-88b2-478a-848d-6cbe361e5e3a None admin] REQUEST_METHOD: `PUT` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:44.421016 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ff2525-88b2-478a-848d-6cbe361e5e3a None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:44.421288 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ff2525-88b2-478a-848d-6cbe361e5e3a None admin] PATH_INFO: `/v3/projects/a28e631793774cb1a575e58d813008e9/users/673ac58961fb445c93eeb8f4665074c0/roles/b870cd1acaa744e1ae1d58b9ac283821` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:44.423199 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ff2525-88b2-478a-848d-6cbe361e5e3a None admin] RBAC: Authorizing `identity:create_grant(project_id=a28e631793774cb1a575e58d813008e9, user_id=673ac58961fb445c93eeb8f4665074c0, role_id=b870cd1acaa744e1ae1d58b9ac283821)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:44.424766 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ff2525-88b2-478a-848d-6cbe361e5e3a None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:44.526624 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 599/1192] 10.222.0.55 () {58 vars in 1439 bytes} [Wed Aug 24 08:36:44 2022] PUT /identity/v3/projects/a28e631793774cb1a575e58d813008e9/users/673ac58961fb445c93eeb8f4665074c0/roles/b870cd1acaa744e1ae1d58b9ac283821 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:36:46.430635 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471a6a80-4411-4e5e-b20d-2710325ae02d None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:46.431155 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471a6a80-4411-4e5e-b20d-2710325ae02d None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:46.431155 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471a6a80-4411-4e5e-b20d-2710325ae02d None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:46.432491 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 594/1193] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:36:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:36:46.438363 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9730494-5906-4b1a-bf30-7e305ae7b324 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:46.438738 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9730494-5906-4b1a-bf30-7e305ae7b324 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:46.438918 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9730494-5906-4b1a-bf30-7e305ae7b324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:46.457574 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-b9730494-5906-4b1a-bf30-7e305ae7b324 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:36:46.458704 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-b9730494-5906-4b1a-bf30-7e305ae7b324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:36:46.524861 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 600/1194] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:36:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:36:46.791975 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471a6a80-4411-4e5e-b20d-2710325ae02d None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:46.841729 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:46.892924 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471a6a80-4411-4e5e-b20d-2710325ae02d None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:46.925560 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c31cf7-a24d-4d74-968c-a639ed93cf4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:46.926172 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c31cf7-a24d-4d74-968c-a639ed93cf4c None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:46.926445 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c31cf7-a24d-4d74-968c-a639ed93cf4c None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:46.926711 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c31cf7-a24d-4d74-968c-a639ed93cf4c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:46.937600 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c31cf7-a24d-4d74-968c-a639ed93cf4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:36:46.942173 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-40c31cf7-a24d-4d74-968c-a639ed93cf4c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:36:46.942173 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:36:46.942173 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:36:46.942173 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:36:46.942173 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:36:46.942173 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:36:46.942173 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:36:46.942173 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:36:46.942173 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:36:46.942173 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:36:46.942173 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:36:46.942173 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:36:46.942173 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:36:46.942173 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:36:46.942173 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:36:46.942173 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:36:46.943259 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:36:46.943259 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:36:46.943259 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:36:46.943259 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:36:46.943259 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:36:46.943259 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:36:46.943259 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:36:46.943259 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:36:46.943259 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:36:46.943259 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:36:46.943259 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:36:46.943259 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:36:46.943259 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:36:46.943259 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:36:46.943259 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:36:46.943259 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:36:46.944340 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:36:46.944340 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:36:46.944340 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:36:46.944340 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:36:46.944340 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:36:46.944340 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:36:46.944340 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:36:46.944340 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:36:46.944340 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:36:46.944340 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:36:46.944340 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:36:46.944340 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:36:46.944340 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 595/1195] 10.222.0.55 () {56 vars in 1096 bytes} [Wed Aug 24 08:36:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:36:46.949352 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9730494-5906-4b1a-bf30-7e305ae7b324 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:47.049081 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9730494-5906-4b1a-bf30-7e305ae7b324 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:47.050517 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:47.082846 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab568d7-0589-4619-bee2-681b9af6dc54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:47.083460 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab568d7-0589-4619-bee2-681b9af6dc54 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:47.083739 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab568d7-0589-4619-bee2-681b9af6dc54 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:47.084012 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab568d7-0589-4619-bee2-681b9af6dc54 None admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:47.084865 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab568d7-0589-4619-bee2-681b9af6dc54 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:47.085175 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab568d7-0589-4619-bee2-681b9af6dc54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:47.086641 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab568d7-0589-4619-bee2-681b9af6dc54 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:47.095368 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 601/1196] 10.222.0.55 () {56 vars in 1099 bytes} [Wed Aug 24 08:36:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:47.101233 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c31cf7-a24d-4d74-968c-a639ed93cf4c None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:47.197442 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c31cf7-a24d-4d74-968c-a639ed93cf4c None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:47.206876 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:47.230816 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1613a0ef-3805-4501-9477-560ae7c9a628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:47.231442 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1613a0ef-3805-4501-9477-560ae7c9a628 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:47.231717 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1613a0ef-3805-4501-9477-560ae7c9a628 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:47.231993 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1613a0ef-3805-4501-9477-560ae7c9a628 None admin] PATH_INFO: `/v3/users/673ac58961fb445c93eeb8f4665074c0` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:47.233740 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1613a0ef-3805-4501-9477-560ae7c9a628 None admin] RBAC: Authorizing `identity:get_user(user_id=673ac58961fb445c93eeb8f4665074c0)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:47.235157 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1613a0ef-3805-4501-9477-560ae7c9a628 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:47.237063 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 596/1197] 10.222.0.55 () {56 vars in 1177 bytes} [Wed Aug 24 08:36:47 2022] GET /identity/v3/users/673ac58961fb445c93eeb8f4665074c0 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:47.242612 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab568d7-0589-4619-bee2-681b9af6dc54 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:47.340354 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab568d7-0589-4619-bee2-681b9af6dc54 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:47.374858 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3755926-b25a-4528-bc36-1dc2fa1f47fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:47.375748 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3755926-b25a-4528-bc36-1dc2fa1f47fc None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:47.376050 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3755926-b25a-4528-bc36-1dc2fa1f47fc None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:47.376318 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3755926-b25a-4528-bc36-1dc2fa1f47fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:47.406892 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:47.445462 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3755926-b25a-4528-bc36-1dc2fa1f47fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:47.447700 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3755926-b25a-4528-bc36-1dc2fa1f47fc None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:47.547317 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 602/1198] 10.222.0.55 () {58 vars in 1282 bytes} [Wed Aug 24 08:36:47 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 08:36:47.553493 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1613a0ef-3805-4501-9477-560ae7c9a628 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:47.667847 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1613a0ef-3805-4501-9477-560ae7c9a628 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:47.700189 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53d6b12-db69-4ab7-b58d-436fab724abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:47.700832 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d6b12-db69-4ab7-b58d-436fab724abd None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:47.701107 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d6b12-db69-4ab7-b58d-436fab724abd None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:47.701909 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d6b12-db69-4ab7-b58d-436fab724abd None admin] PATH_INFO: `/v3/projects/a28e631793774cb1a575e58d813008e9` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:47.703025 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53d6b12-db69-4ab7-b58d-436fab724abd None admin] RBAC: Authorizing `identity:get_project(project_id=a28e631793774cb1a575e58d813008e9)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:47.704446 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53d6b12-db69-4ab7-b58d-436fab724abd None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:47.706083 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 597/1199] 10.222.0.55 () {56 vars in 1186 bytes} [Wed Aug 24 08:36:47 2022] GET /identity/v3/projects/a28e631793774cb1a575e58d813008e9 => generated 361 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:47.712215 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3755926-b25a-4528-bc36-1dc2fa1f47fc None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:36:47.772417 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:47.809276 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3755926-b25a-4528-bc36-1dc2fa1f47fc None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:36:47.843612 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109b708e-c889-4e28-a6bf-7b8d2c67dda3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:36:47.844207 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109b708e-c889-4e28-a6bf-7b8d2c67dda3 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:36:47.844479 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109b708e-c889-4e28-a6bf-7b8d2c67dda3 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:36:47.844774 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109b708e-c889-4e28-a6bf-7b8d2c67dda3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:36:47.845488 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109b708e-c889-4e28-a6bf-7b8d2c67dda3 None admin] RBAC: Adding query filter params (role.id=b870cd1acaa744e1ae1d58b9ac283821, scope.project.id=a28e631793774cb1a575e58d813008e9, user.id=673ac58961fb445c93eeb8f4665074c0) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:36:47.845789 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109b708e-c889-4e28-a6bf-7b8d2c67dda3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:36:47.847194 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109b708e-c889-4e28-a6bf-7b8d2c67dda3 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:36:47.852968 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 603/1200] 10.222.0.55 () {56 vars in 1374 bytes} [Wed Aug 24 08:36:47 2022] GET /identity/v3/role_assignments?user.id=673ac58961fb445c93eeb8f4665074c0&scope.project.id=a28e631793774cb1a575e58d813008e9&role.id=b870cd1acaa744e1ae1d58b9ac283821 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:36:57.696039 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=78 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:36:57.850371 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=36 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:38:07.513992 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df985d7d-0d4c-463e-8afd-68300b07b61f None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:07.514547 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df985d7d-0d4c-463e-8afd-68300b07b61f None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:07.514547 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df985d7d-0d4c-463e-8afd-68300b07b61f None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:07.516095 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 598/1201] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:38:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:38:07.521595 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f704530-a530-464b-b4d7-79c14636f9fe None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:07.522032 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f704530-a530-464b-b4d7-79c14636f9fe None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:07.522123 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f704530-a530-464b-b4d7-79c14636f9fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:07.540921 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-4f704530-a530-464b-b4d7-79c14636f9fe None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:38:07.542163 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-4f704530-a530-464b-b4d7-79c14636f9fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:38:07.614108 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 604/1202] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:38:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:38:07.888007 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df985d7d-0d4c-463e-8afd-68300b07b61f None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:07.991432 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df985d7d-0d4c-463e-8afd-68300b07b61f None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:08.027083 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2629484-8d9c-4f63-a529-80c84ba79465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:08.030274 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2629484-8d9c-4f63-a529-80c84ba79465 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:08.030274 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2629484-8d9c-4f63-a529-80c84ba79465 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:08.030274 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2629484-8d9c-4f63-a529-80c84ba79465 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:08.038824 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2629484-8d9c-4f63-a529-80c84ba79465 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:38:08.040252 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2629484-8d9c-4f63-a529-80c84ba79465 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:38:08.054332 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-a2629484-8d9c-4f63-a529-80c84ba79465 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:38:08.054332 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:38:08.054332 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:38:08.054332 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:38:08.054332 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:38:08.054332 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:38:08.054332 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:38:08.054332 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:38:08.054332 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:38:08.054332 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:38:08.054332 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:38:08.054332 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:38:08.054332 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:38:08.054332 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:38:08.054332 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:38:08.054332 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:38:08.055678 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:38:08.055678 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:38:08.055678 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:38:08.055678 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:38:08.055678 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:38:08.055678 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:38:08.055678 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:38:08.055678 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:38:08.055678 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:38:08.055678 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:38:08.055678 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:38:08.055678 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:38:08.055678 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:38:08.055678 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:38:08.055678 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:38:08.055678 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:38:08.055678 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:38:08.057076 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:38:08.057076 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:38:08.057076 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:38:08.057076 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 599/1203] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:38:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:38:08.061463 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f704530-a530-464b-b4d7-79c14636f9fe None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:08.162552 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f704530-a530-464b-b4d7-79c14636f9fe None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:08.193349 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:38:08.199206 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ed17ba-cf4e-4873-8e0d-7571ba7ffbbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:08.199820 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ed17ba-cf4e-4873-8e0d-7571ba7ffbbd None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:08.200102 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ed17ba-cf4e-4873-8e0d-7571ba7ffbbd None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:08.200371 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ed17ba-cf4e-4873-8e0d-7571ba7ffbbd None admin] PATH_INFO: `/v3/domains` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:08.201063 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ed17ba-cf4e-4873-8e0d-7571ba7ffbbd None admin] RBAC: Adding query filter params (name=Default) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:38:08.207376 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ed17ba-cf4e-4873-8e0d-7571ba7ffbbd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:38:08.207376 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ed17ba-cf4e-4873-8e0d-7571ba7ffbbd None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:38:08.214063 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 605/1204] 10.222.0.55 () {56 vars in 1109 bytes} [Wed Aug 24 08:38:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:38:08.220429 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2629484-8d9c-4f63-a529-80c84ba79465 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:08.274422 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:38:08.320787 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2629484-8d9c-4f63-a529-80c84ba79465 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:08.353568 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060688c9-427d-490b-b2ec-8f21539f7812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:08.354204 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060688c9-427d-490b-b2ec-8f21539f7812 None admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:08.354477 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060688c9-427d-490b-b2ec-8f21539f7812 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:08.354746 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060688c9-427d-490b-b2ec-8f21539f7812 None admin] PATH_INFO: `/v3/users` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:08.355378 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060688c9-427d-490b-b2ec-8f21539f7812 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:38:08.356828 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060688c9-427d-490b-b2ec-8f21539f7812 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:38:08.446064 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 600/1205] 10.222.0.55 () {60 vars in 1132 bytes} [Wed Aug 24 08:38:08 2022] POST /identity/v3/users => generated 251 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:38:10.460577 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029a2beb-96c9-4df4-8608-02c0f4fe0272 None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:10.461153 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029a2beb-96c9-4df4-8608-02c0f4fe0272 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:10.461153 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029a2beb-96c9-4df4-8608-02c0f4fe0272 None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:10.462395 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 606/1206] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:38:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:38:10.467763 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bbba20-cd9f-4f6d-b726-447984ffe61a None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:10.468694 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bbba20-cd9f-4f6d-b726-447984ffe61a None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:10.468877 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bbba20-cd9f-4f6d-b726-447984ffe61a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:10.488286 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-12bbba20-cd9f-4f6d-b726-447984ffe61a None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:38:10.489518 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-12bbba20-cd9f-4f6d-b726-447984ffe61a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:38:10.556708 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 601/1207] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:38:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:38:10.825878 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029a2beb-96c9-4df4-8608-02c0f4fe0272 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:10.937409 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029a2beb-96c9-4df4-8608-02c0f4fe0272 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:10.970375 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682ba4de-f83c-471f-b2d4-e0cb808420a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:10.970997 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682ba4de-f83c-471f-b2d4-e0cb808420a1 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:10.971269 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682ba4de-f83c-471f-b2d4-e0cb808420a1 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:10.971533 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682ba4de-f83c-471f-b2d4-e0cb808420a1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:10.982151 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682ba4de-f83c-471f-b2d4-e0cb808420a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:38:10.987276 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-682ba4de-f83c-471f-b2d4-e0cb808420a1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:38:10.987276 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:38:10.987276 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:38:10.987276 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:38:10.987276 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:38:10.987276 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:38:10.987276 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:38:10.987276 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:38:10.987276 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:38:10.987276 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:38:10.987276 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:38:10.987276 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:38:10.987276 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:38:10.987276 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:38:10.987276 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:38:10.987276 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:38:10.988477 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:38:10.988477 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:38:10.988477 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:38:10.988477 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:38:10.988477 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:38:10.988477 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:38:10.988477 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:38:10.988477 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:38:10.988477 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:38:10.988477 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:38:10.988477 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:38:10.988477 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:38:10.988477 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:38:10.988477 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:38:10.988477 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:38:10.988477 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:38:10.989451 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:38:10.989451 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:38:10.989451 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:38:10.989451 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:38:10.989451 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:38:10.989451 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:38:10.989451 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:38:10.989451 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:38:10.989451 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:38:10.989451 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:38:10.989451 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:38:10.989451 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:38:10.989451 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 607/1208] 10.222.0.55 () {56 vars in 1102 bytes} [Wed Aug 24 08:38:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:38:10.994513 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bbba20-cd9f-4f6d-b726-447984ffe61a None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:11.034435 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:38:11.093507 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bbba20-cd9f-4f6d-b726-447984ffe61a None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:11.127274 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea64a1e-75de-4759-ba41-dec45ff2f061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:11.127869 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea64a1e-75de-4759-ba41-dec45ff2f061 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:11.128149 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea64a1e-75de-4759-ba41-dec45ff2f061 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:11.128423 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea64a1e-75de-4759-ba41-dec45ff2f061 None admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:11.129120 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea64a1e-75de-4759-ba41-dec45ff2f061 None admin] RBAC: Adding query filter params (name=service) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:38:11.129428 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea64a1e-75de-4759-ba41-dec45ff2f061 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:38:11.130840 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea64a1e-75de-4759-ba41-dec45ff2f061 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:38:11.138638 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 602/1209] 10.222.0.55 () {56 vars in 1103 bytes} [Wed Aug 24 08:38:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:38:11.144859 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682ba4de-f83c-471f-b2d4-e0cb808420a1 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:11.194753 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:38:11.248052 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682ba4de-f83c-471f-b2d4-e0cb808420a1 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:11.283614 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37025fd4-d4c3-4162-830e-2e3299dbe5bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:11.285415 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37025fd4-d4c3-4162-830e-2e3299dbe5bc None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:11.285415 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37025fd4-d4c3-4162-830e-2e3299dbe5bc None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:11.285415 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37025fd4-d4c3-4162-830e-2e3299dbe5bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:11.295655 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37025fd4-d4c3-4162-830e-2e3299dbe5bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:38:11.297182 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37025fd4-d4c3-4162-830e-2e3299dbe5bc None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:38:11.313033 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-37025fd4-d4c3-4162-830e-2e3299dbe5bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:38:11.313033 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:38:11.313033 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:38:11.313033 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:38:11.313033 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:38:11.313033 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:38:11.313033 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:38:11.313033 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:38:11.313033 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:38:11.313033 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:38:11.313033 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:38:11.313033 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:38:11.313033 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:38:11.313033 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:38:11.313033 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:38:11.313033 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:38:11.315559 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:38:11.315559 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:38:11.315559 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:38:11.315559 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:38:11.315559 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:38:11.315559 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:38:11.315559 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:38:11.315559 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:38:11.315559 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:38:11.315559 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:38:11.315559 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:38:11.315559 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:38:11.315559 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:38:11.315559 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:38:11.315559 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:38:11.315559 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:38:11.315559 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:38:11.316549 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:38:11.316549 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:38:11.316549 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:38:11.316549 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 608/1210] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:38:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:38:11.320320 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea64a1e-75de-4759-ba41-dec45ff2f061 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:11.418623 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:38:11.421554 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea64a1e-75de-4759-ba41-dec45ff2f061 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:11.455137 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78536f3e-face-427b-ba26-c31bda6b12e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:11.455747 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78536f3e-face-427b-ba26-c31bda6b12e0 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:11.456026 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78536f3e-face-427b-ba26-c31bda6b12e0 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:11.456297 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78536f3e-face-427b-ba26-c31bda6b12e0 None admin] PATH_INFO: `/v3/domains` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:11.456978 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78536f3e-face-427b-ba26-c31bda6b12e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:38:11.457284 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78536f3e-face-427b-ba26-c31bda6b12e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:38:11.458703 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78536f3e-face-427b-ba26-c31bda6b12e0 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:38:11.470187 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 603/1211] 10.222.0.55 () {56 vars in 1109 bytes} [Wed Aug 24 08:38:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:38:11.475968 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37025fd4-d4c3-4162-830e-2e3299dbe5bc None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:11.587299 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37025fd4-d4c3-4162-830e-2e3299dbe5bc None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:11.589131 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:38:11.641354 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30047f99-3d97-4b11-9eb9-7ad04ef21584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:11.642399 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30047f99-3d97-4b11-9eb9-7ad04ef21584 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:11.642889 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30047f99-3d97-4b11-9eb9-7ad04ef21584 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:11.643364 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30047f99-3d97-4b11-9eb9-7ad04ef21584 None admin] PATH_INFO: `/v3/users/placement` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:11.658158 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30047f99-3d97-4b11-9eb9-7ad04ef21584 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:38:11.663058 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30047f99-3d97-4b11-9eb9-7ad04ef21584 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:38:11.682951 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-30047f99-3d97-4b11-9eb9-7ad04ef21584 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 08:38:11.682951 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:38:11.682951 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:38:11.682951 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:38:11.682951 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:38:11.682951 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:38:11.682951 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:38:11.682951 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:38:11.682951 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:38:11.682951 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:38:11.682951 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:38:11.682951 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:38:11.682951 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:38:11.682951 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:38:11.682951 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:38:11.682951 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:38:11.684137 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:38:11.684137 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:38:11.684137 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:38:11.684137 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:38:11.684137 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:38:11.684137 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:38:11.684137 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:38:11.684137 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:38:11.684137 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:38:11.684137 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:38:11.684137 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:38:11.684137 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:38:11.684137 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:38:11.684137 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:38:11.684137 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:38:11.684137 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:38:11.684137 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:38:11.684137 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:38:11.685152 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:38:11.685152 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:38:11.685152 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:38:11.685152 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:38:11.685152 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:38:11.685152 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:38:11.685152 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:38:11.685152 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:38:11.685152 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 08:38:11.685152 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:38:11.685152 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 609/1212] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:38:11 2022] GET /identity/v3/users/placement => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:38:11.694092 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78536f3e-face-427b-ba26-c31bda6b12e0 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:11.796310 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78536f3e-face-427b-ba26-c31bda6b12e0 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:11.830635 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a290461-5a79-4208-8ab5-2b5372557a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:11.831245 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a290461-5a79-4208-8ab5-2b5372557a6f None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:11.831525 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a290461-5a79-4208-8ab5-2b5372557a6f None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:11.831797 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a290461-5a79-4208-8ab5-2b5372557a6f None admin] PATH_INFO: `/v3/users` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:11.832531 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a290461-5a79-4208-8ab5-2b5372557a6f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:38:11.832860 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a290461-5a79-4208-8ab5-2b5372557a6f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:38:11.834299 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a290461-5a79-4208-8ab5-2b5372557a6f None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:38:11.852900 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 604/1213] 10.222.0.55 () {56 vars in 1143 bytes} [Wed Aug 24 08:38:11 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:38:11.858777 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30047f99-3d97-4b11-9eb9-7ad04ef21584 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:11.962309 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30047f99-3d97-4b11-9eb9-7ad04ef21584 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:11.996476 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50edda59-5052-42fa-8c66-fc9ac2db6e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:11.997121 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50edda59-5052-42fa-8c66-fc9ac2db6e19 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:11.997420 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50edda59-5052-42fa-8c66-fc9ac2db6e19 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:11.997696 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50edda59-5052-42fa-8c66-fc9ac2db6e19 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:12.032938 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:38:12.071667 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50edda59-5052-42fa-8c66-fc9ac2db6e19 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:38:12.073217 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50edda59-5052-42fa-8c66-fc9ac2db6e19 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:38:12.173943 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 610/1214] 10.222.0.55 () {58 vars in 1282 bytes} [Wed Aug 24 08:38:11 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 08:38:12.180241 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a290461-5a79-4208-8ab5-2b5372557a6f None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:12.203596 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:38:12.286099 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a290461-5a79-4208-8ab5-2b5372557a6f None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:12.322487 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8653aa5f-6b07-49b8-b83c-0cc02d80abee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:12.323123 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8653aa5f-6b07-49b8-b83c-0cc02d80abee None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:12.323425 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8653aa5f-6b07-49b8-b83c-0cc02d80abee None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:12.323695 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8653aa5f-6b07-49b8-b83c-0cc02d80abee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:12.333895 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8653aa5f-6b07-49b8-b83c-0cc02d80abee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:38:12.335389 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8653aa5f-6b07-49b8-b83c-0cc02d80abee None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:38:12.351208 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-8653aa5f-6b07-49b8-b83c-0cc02d80abee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:38:12.351208 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:38:12.351208 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:38:12.351208 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:38:12.351208 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:38:12.351208 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:38:12.351208 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:38:12.351208 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:38:12.351208 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:38:12.351208 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:38:12.351208 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:38:12.351208 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:38:12.351208 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:38:12.351208 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:38:12.351208 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:38:12.351208 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:38:12.357815 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:38:12.357815 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:38:12.357815 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:38:12.357815 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:38:12.357815 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:38:12.357815 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:38:12.357815 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:38:12.357815 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:38:12.357815 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:38:12.357815 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:38:12.357815 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:38:12.357815 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:38:12.357815 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:38:12.357815 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:38:12.357815 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:38:12.357815 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:38:12.357815 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:38:12.359466 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:38:12.359466 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:38:12.359466 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:38:12.359466 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 605/1215] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:38:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:38:12.359759 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50edda59-5052-42fa-8c66-fc9ac2db6e19 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:12.422870 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:38:12.461619 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50edda59-5052-42fa-8c66-fc9ac2db6e19 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:12.495150 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4cbced-d703-45a8-9a1f-19b4cafaf129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:12.495803 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4cbced-d703-45a8-9a1f-19b4cafaf129 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:12.496007 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4cbced-d703-45a8-9a1f-19b4cafaf129 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:12.496272 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4cbced-d703-45a8-9a1f-19b4cafaf129 None admin] PATH_INFO: `/v3/domains` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:12.496933 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4cbced-d703-45a8-9a1f-19b4cafaf129 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:38:12.497228 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4cbced-d703-45a8-9a1f-19b4cafaf129 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:38:12.498746 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4cbced-d703-45a8-9a1f-19b4cafaf129 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:38:12.510316 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 611/1216] 10.222.0.55 () {56 vars in 1109 bytes} [Wed Aug 24 08:38:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:38:12.516446 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8653aa5f-6b07-49b8-b83c-0cc02d80abee None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:12.585786 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:38:12.616628 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8653aa5f-6b07-49b8-b83c-0cc02d80abee None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:12.650771 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9239f01-bdce-4871-aa49-6d02ced13a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:12.651398 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9239f01-bdce-4871-aa49-6d02ced13a5b None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:12.651678 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9239f01-bdce-4871-aa49-6d02ced13a5b None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:12.651952 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9239f01-bdce-4871-aa49-6d02ced13a5b None admin] PATH_INFO: `/v3/projects/service` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:12.659267 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9239f01-bdce-4871-aa49-6d02ced13a5b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:38:12.665575 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9239f01-bdce-4871-aa49-6d02ced13a5b None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:38:12.673897 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-e9239f01-bdce-4871-aa49-6d02ced13a5b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:38:12.673897 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:38:12.673897 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:38:12.673897 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:38:12.673897 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:38:12.673897 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:38:12.673897 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:38:12.673897 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:38:12.673897 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:38:12.673897 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:38:12.673897 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:38:12.673897 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:38:12.673897 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:38:12.673897 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:38:12.673897 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:38:12.674989 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:38:12.674989 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:38:12.674989 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:38:12.674989 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:38:12.674989 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:38:12.674989 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:38:12.674989 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:38:12.674989 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:38:12.674989 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:38:12.674989 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:38:12.674989 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:38:12.674989 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:38:12.674989 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:38:12.674989 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:38:12.674989 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:38:12.674989 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:38:12.674989 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:38:12.676129 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:38:12.676129 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:38:12.676129 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:38:12.676129 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:38:12.676129 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:38:12.676129 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:38:12.676129 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:38:12.676129 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:38:12.676129 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 606/1217] 10.222.0.55 () {56 vars in 1111 bytes} [Wed Aug 24 08:38:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:38:12.681355 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4cbced-d703-45a8-9a1f-19b4cafaf129 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:12.784925 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4cbced-d703-45a8-9a1f-19b4cafaf129 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:12.807653 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:38:12.819965 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a8d21f-4115-45c7-9ad3-15783f92c5dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:12.820949 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a8d21f-4115-45c7-9ad3-15783f92c5dd None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:12.825879 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a8d21f-4115-45c7-9ad3-15783f92c5dd None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:12.825879 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a8d21f-4115-45c7-9ad3-15783f92c5dd None admin] PATH_INFO: `/v3/projects` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:12.825879 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a8d21f-4115-45c7-9ad3-15783f92c5dd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:38:12.825879 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a8d21f-4115-45c7-9ad3-15783f92c5dd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:38:12.825879 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a8d21f-4115-45c7-9ad3-15783f92c5dd None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:38:12.839274 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 612/1218] 10.222.0.55 () {56 vars in 1148 bytes} [Wed Aug 24 08:38:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:38:12.845352 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9239f01-bdce-4871-aa49-6d02ced13a5b None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:12.949527 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9239f01-bdce-4871-aa49-6d02ced13a5b None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:12.978479 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:38:12.985404 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cbc5f0-a298-4511-89c0-ed9061b814ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:12.986292 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cbc5f0-a298-4511-89c0-ed9061b814ae None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:12.986701 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cbc5f0-a298-4511-89c0-ed9061b814ae None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:12.987141 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cbc5f0-a298-4511-89c0-ed9061b814ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:12.987864 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cbc5f0-a298-4511-89c0-ed9061b814ae None admin] RBAC: Adding query filter params (role.id=bab90e02eebc473e9b911c8660ed9b76, scope.project.id=420b18725f2342159646fb86438318fd, user.id=cf0c99771661413aaa0f237af6b394ab) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:38:12.988173 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cbc5f0-a298-4511-89c0-ed9061b814ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:38:12.989646 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cbc5f0-a298-4511-89c0-ed9061b814ae None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:38:12.996567 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 607/1219] 10.222.0.55 () {56 vars in 1374 bytes} [Wed Aug 24 08:38:12 2022] GET /identity/v3/role_assignments?user.id=cf0c99771661413aaa0f237af6b394ab&scope.project.id=420b18725f2342159646fb86438318fd&role.id=bab90e02eebc473e9b911c8660ed9b76 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:38:14.930819 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f4d45f-e2e3-4f7c-926a-99df620e6117 None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:14.931411 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f4d45f-e2e3-4f7c-926a-99df620e6117 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:14.931411 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f4d45f-e2e3-4f7c-926a-99df620e6117 None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:14.932737 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 613/1220] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:38:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:38:14.937879 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582b4f41-b508-4e23-a2d6-d5adf1a71206 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:14.938294 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582b4f41-b508-4e23-a2d6-d5adf1a71206 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:14.938446 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582b4f41-b508-4e23-a2d6-d5adf1a71206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:14.956347 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-582b4f41-b508-4e23-a2d6-d5adf1a71206 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:38:14.957566 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-582b4f41-b508-4e23-a2d6-d5adf1a71206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:38:15.022284 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 608/1221] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:38:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:38:15.292911 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f4d45f-e2e3-4f7c-926a-99df620e6117 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:15.395124 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f4d45f-e2e3-4f7c-926a-99df620e6117 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:15.428327 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e73124-d42c-4fdf-b3eb-936c6584921e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:15.429003 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e73124-d42c-4fdf-b3eb-936c6584921e None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:15.429264 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e73124-d42c-4fdf-b3eb-936c6584921e None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:15.429532 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e73124-d42c-4fdf-b3eb-936c6584921e None admin] PATH_INFO: `/v3/roles/service` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:15.440426 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e73124-d42c-4fdf-b3eb-936c6584921e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:38:15.445587 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-90e73124-d42c-4fdf-b3eb-936c6584921e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:38:15.445587 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:38:15.445587 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:38:15.445587 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:38:15.445587 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:38:15.445587 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:38:15.445587 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:38:15.445587 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:38:15.445587 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:38:15.445587 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:38:15.445587 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:38:15.445587 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:38:15.445587 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:38:15.445587 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:38:15.445587 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:38:15.445587 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:38:15.450913 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:38:15.450913 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:38:15.450913 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:38:15.450913 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:38:15.450913 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:38:15.450913 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:38:15.450913 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:38:15.450913 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:38:15.450913 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:38:15.450913 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:38:15.450913 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:38:15.450913 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:38:15.450913 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:38:15.450913 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:38:15.450913 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:38:15.450913 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:38:15.452124 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:38:15.452124 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:38:15.452124 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:38:15.452124 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:38:15.452124 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:38:15.452124 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:38:15.452124 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:38:15.452124 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:38:15.452124 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:38:15.452124 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:38:15.452124 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:38:15.452124 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:38:15.452124 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 614/1222] 10.222.0.55 () {56 vars in 1102 bytes} [Wed Aug 24 08:38:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:38:15.453099 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582b4f41-b508-4e23-a2d6-d5adf1a71206 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:15.554012 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582b4f41-b508-4e23-a2d6-d5adf1a71206 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:15.578175 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:38:15.588777 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c93a91-69b5-4661-9954-9704dbb3ae26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:15.593218 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c93a91-69b5-4661-9954-9704dbb3ae26 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:15.593218 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c93a91-69b5-4661-9954-9704dbb3ae26 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:15.593218 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c93a91-69b5-4661-9954-9704dbb3ae26 None admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:15.593218 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c93a91-69b5-4661-9954-9704dbb3ae26 None admin] RBAC: Adding query filter params (name=service) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:38:15.593218 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c93a91-69b5-4661-9954-9704dbb3ae26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:38:15.593218 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c93a91-69b5-4661-9954-9704dbb3ae26 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:38:15.600556 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 609/1223] 10.222.0.55 () {56 vars in 1103 bytes} [Wed Aug 24 08:38:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:38:15.606571 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e73124-d42c-4fdf-b3eb-936c6584921e None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:15.651320 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:38:15.707132 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e73124-d42c-4fdf-b3eb-936c6584921e None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:15.740998 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909d53cb-c6b0-4682-89c8-e26c068563f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:15.741620 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909d53cb-c6b0-4682-89c8-e26c068563f0 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:15.741895 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909d53cb-c6b0-4682-89c8-e26c068563f0 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:15.742164 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909d53cb-c6b0-4682-89c8-e26c068563f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:15.752656 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909d53cb-c6b0-4682-89c8-e26c068563f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:38:15.754085 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909d53cb-c6b0-4682-89c8-e26c068563f0 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:38:15.768810 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-909d53cb-c6b0-4682-89c8-e26c068563f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:38:15.768810 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:38:15.768810 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:38:15.768810 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:38:15.768810 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:38:15.768810 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:38:15.768810 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:38:15.768810 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:38:15.768810 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:38:15.768810 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:38:15.768810 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:38:15.768810 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:38:15.768810 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:38:15.768810 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:38:15.768810 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:38:15.768810 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:38:15.770012 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:38:15.770012 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:38:15.770012 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:38:15.770012 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:38:15.770012 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:38:15.770012 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:38:15.770012 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:38:15.770012 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:38:15.770012 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:38:15.770012 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:38:15.770012 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:38:15.770012 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:38:15.770012 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:38:15.770012 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:38:15.770012 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:38:15.770012 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:38:15.770012 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:38:15.771157 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:38:15.771157 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:38:15.771157 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:38:15.771157 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 615/1224] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:38:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:38:15.775928 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c93a91-69b5-4661-9954-9704dbb3ae26 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:15.885944 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c93a91-69b5-4661-9954-9704dbb3ae26 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:15.921708 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27b0dd6-26a2-405c-a9c8-43d02b3e35b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:15.933073 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27b0dd6-26a2-405c-a9c8-43d02b3e35b9 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:15.933073 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27b0dd6-26a2-405c-a9c8-43d02b3e35b9 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:15.933073 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27b0dd6-26a2-405c-a9c8-43d02b3e35b9 None admin] PATH_INFO: `/v3/domains` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:15.933073 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27b0dd6-26a2-405c-a9c8-43d02b3e35b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:38:15.933073 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27b0dd6-26a2-405c-a9c8-43d02b3e35b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:38:15.933073 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27b0dd6-26a2-405c-a9c8-43d02b3e35b9 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:38:15.938788 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 610/1225] 10.222.0.55 () {56 vars in 1109 bytes} [Wed Aug 24 08:38:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:38:15.944754 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909d53cb-c6b0-4682-89c8-e26c068563f0 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:16.041999 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:38:16.048913 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909d53cb-c6b0-4682-89c8-e26c068563f0 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:16.083668 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eadee4c-b2b0-4ba7-be9d-1b0e54a2f3e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:16.089099 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eadee4c-b2b0-4ba7-be9d-1b0e54a2f3e9 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:16.089099 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eadee4c-b2b0-4ba7-be9d-1b0e54a2f3e9 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:16.089099 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eadee4c-b2b0-4ba7-be9d-1b0e54a2f3e9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:16.092650 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eadee4c-b2b0-4ba7-be9d-1b0e54a2f3e9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:38:16.094123 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eadee4c-b2b0-4ba7-be9d-1b0e54a2f3e9 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:38:16.105150 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-8eadee4c-b2b0-4ba7-be9d-1b0e54a2f3e9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 08:38:16.105150 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:38:16.105150 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:38:16.105150 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:38:16.105150 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:38:16.105150 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:38:16.105150 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:38:16.105150 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:38:16.105150 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:38:16.105150 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:38:16.105150 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:38:16.105150 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:38:16.105150 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:38:16.105150 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:38:16.105150 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:38:16.105150 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:38:16.106420 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:38:16.106420 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:38:16.106420 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:38:16.106420 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:38:16.106420 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:38:16.106420 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:38:16.106420 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:38:16.106420 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:38:16.106420 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:38:16.106420 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:38:16.106420 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:38:16.106420 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:38:16.106420 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:38:16.106420 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:38:16.106420 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:38:16.106420 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:38:16.106420 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:38:16.106420 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:38:16.107434 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:38:16.107434 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:38:16.107434 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:38:16.107434 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:38:16.107434 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:38:16.107434 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:38:16.107434 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:38:16.107434 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:38:16.107434 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 08:38:16.107434 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:38:16.107434 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 616/1226] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:38:15 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:38:16.112938 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27b0dd6-26a2-405c-a9c8-43d02b3e35b9 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:16.160902 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:38:16.217646 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27b0dd6-26a2-405c-a9c8-43d02b3e35b9 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:16.252598 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d76e00-d153-4206-9711-c8bc1fca8990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:16.255080 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d76e00-d153-4206-9711-c8bc1fca8990 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:16.255080 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d76e00-d153-4206-9711-c8bc1fca8990 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:16.255080 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d76e00-d153-4206-9711-c8bc1fca8990 None admin] PATH_INFO: `/v3/users` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:16.255080 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d76e00-d153-4206-9711-c8bc1fca8990 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:38:16.255080 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d76e00-d153-4206-9711-c8bc1fca8990 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:38:16.256458 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d76e00-d153-4206-9711-c8bc1fca8990 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:38:16.274332 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 611/1227] 10.222.0.55 () {56 vars in 1143 bytes} [Wed Aug 24 08:38:16 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:38:16.280524 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eadee4c-b2b0-4ba7-be9d-1b0e54a2f3e9 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:16.391696 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eadee4c-b2b0-4ba7-be9d-1b0e54a2f3e9 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:16.428355 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c26f41-80e4-4317-8b1d-904367f27561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:16.430307 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c26f41-80e4-4317-8b1d-904367f27561 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:16.430307 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c26f41-80e4-4317-8b1d-904367f27561 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:16.430307 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c26f41-80e4-4317-8b1d-904367f27561 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:16.441793 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c26f41-80e4-4317-8b1d-904367f27561 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:38:16.443378 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c26f41-80e4-4317-8b1d-904367f27561 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:38:16.459028 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-c7c26f41-80e4-4317-8b1d-904367f27561 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:38:16.459028 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:38:16.459028 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:38:16.459028 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:38:16.459028 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:38:16.459028 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:38:16.459028 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:38:16.459028 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:38:16.459028 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:38:16.459028 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:38:16.459028 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:38:16.459028 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:38:16.459028 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:38:16.459028 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:38:16.459028 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:38:16.459028 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:38:16.460294 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:38:16.460294 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:38:16.460294 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:38:16.460294 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:38:16.460294 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:38:16.460294 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:38:16.460294 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:38:16.460294 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:38:16.460294 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:38:16.460294 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:38:16.460294 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:38:16.460294 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:38:16.460294 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:38:16.460294 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:38:16.460294 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:38:16.460294 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:38:16.460294 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:38:16.461444 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:38:16.461444 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:38:16.461444 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:38:16.461444 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 617/1228] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:38:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:38:16.467180 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d76e00-d153-4206-9711-c8bc1fca8990 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:16.583235 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d76e00-d153-4206-9711-c8bc1fca8990 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:16.585743 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:38:16.618856 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3f4ca8-4758-4e59-8685-35ccd3f4b0b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:16.619464 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3f4ca8-4758-4e59-8685-35ccd3f4b0b3 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:16.619741 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3f4ca8-4758-4e59-8685-35ccd3f4b0b3 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:16.620008 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3f4ca8-4758-4e59-8685-35ccd3f4b0b3 None admin] PATH_INFO: `/v3/domains` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:16.620677 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3f4ca8-4758-4e59-8685-35ccd3f4b0b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:38:16.621137 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3f4ca8-4758-4e59-8685-35ccd3f4b0b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:38:16.622505 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3f4ca8-4758-4e59-8685-35ccd3f4b0b3 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:38:16.635138 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 612/1229] 10.222.0.55 () {56 vars in 1109 bytes} [Wed Aug 24 08:38:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:38:16.644581 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c26f41-80e4-4317-8b1d-904367f27561 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:16.654400 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:38:16.752993 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c26f41-80e4-4317-8b1d-904367f27561 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:16.788204 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dff900-2e2a-496d-a9bc-e7e142a759bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:16.788884 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dff900-2e2a-496d-a9bc-e7e142a759bb None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:16.789159 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dff900-2e2a-496d-a9bc-e7e142a759bb None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:16.789431 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dff900-2e2a-496d-a9bc-e7e142a759bb None admin] PATH_INFO: `/v3/projects/service` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:16.797054 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dff900-2e2a-496d-a9bc-e7e142a759bb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:38:16.798508 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dff900-2e2a-496d-a9bc-e7e142a759bb None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:38:16.810376 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-89dff900-2e2a-496d-a9bc-e7e142a759bb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:38:16.810376 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:38:16.810376 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:38:16.810376 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:38:16.810376 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:38:16.810376 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:38:16.810376 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:38:16.810376 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:38:16.810376 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:38:16.810376 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:38:16.810376 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:38:16.810376 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:38:16.810376 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:38:16.810376 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:38:16.810376 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:38:16.811542 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:38:16.811542 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:38:16.811542 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:38:16.811542 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:38:16.811542 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:38:16.811542 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:38:16.811542 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:38:16.811542 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:38:16.811542 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:38:16.811542 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:38:16.811542 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:38:16.811542 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:38:16.811542 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:38:16.811542 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:38:16.811542 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:38:16.811542 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:38:16.811542 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:38:16.813290 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:38:16.813290 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:38:16.813290 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:38:16.813290 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:38:16.813290 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:38:16.813290 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:38:16.813290 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:38:16.813290 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:38:16.813290 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 618/1230] 10.222.0.55 () {56 vars in 1111 bytes} [Wed Aug 24 08:38:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:38:16.818248 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3f4ca8-4758-4e59-8685-35ccd3f4b0b3 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:16.926670 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3f4ca8-4758-4e59-8685-35ccd3f4b0b3 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:16.970106 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf9b6a1-b3b3-46af-937a-dca128d46cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:16.970824 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf9b6a1-b3b3-46af-937a-dca128d46cf6 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:16.970988 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf9b6a1-b3b3-46af-937a-dca128d46cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:16.971262 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf9b6a1-b3b3-46af-937a-dca128d46cf6 None admin] PATH_INFO: `/v3/projects` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:16.971961 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf9b6a1-b3b3-46af-937a-dca128d46cf6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:38:16.972268 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf9b6a1-b3b3-46af-937a-dca128d46cf6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:38:16.973822 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf9b6a1-b3b3-46af-937a-dca128d46cf6 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:38:16.990531 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 613/1231] 10.222.0.55 () {56 vars in 1148 bytes} [Wed Aug 24 08:38:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:38:16.997094 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dff900-2e2a-496d-a9bc-e7e142a759bb None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:17.068504 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:38:17.106043 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dff900-2e2a-496d-a9bc-e7e142a759bb None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:17.143840 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5f4d6e-34a9-4b0c-a8e8-6948700ac944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:17.144595 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5f4d6e-34a9-4b0c-a8e8-6948700ac944 None admin] REQUEST_METHOD: `PUT` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:17.144991 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5f4d6e-34a9-4b0c-a8e8-6948700ac944 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:17.145354 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5f4d6e-34a9-4b0c-a8e8-6948700ac944 None admin] PATH_INFO: `/v3/projects/420b18725f2342159646fb86438318fd/users/cf0c99771661413aaa0f237af6b394ab/roles/bab90e02eebc473e9b911c8660ed9b76` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:17.163885 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5f4d6e-34a9-4b0c-a8e8-6948700ac944 None admin] RBAC: Authorizing `identity:create_grant(project_id=420b18725f2342159646fb86438318fd, user_id=cf0c99771661413aaa0f237af6b394ab, role_id=bab90e02eebc473e9b911c8660ed9b76)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:38:17.174228 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5f4d6e-34a9-4b0c-a8e8-6948700ac944 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:38:17.306813 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 619/1232] 10.222.0.55 () {58 vars in 1439 bytes} [Wed Aug 24 08:38:16 2022] PUT /identity/v3/projects/420b18725f2342159646fb86438318fd/users/cf0c99771661413aaa0f237af6b394ab/roles/bab90e02eebc473e9b911c8660ed9b76 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:38:19.271450 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98d3122-e2c3-4ac0-acaf-3fb8e3c3c40c None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:19.272005 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98d3122-e2c3-4ac0-acaf-3fb8e3c3c40c None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:19.272005 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98d3122-e2c3-4ac0-acaf-3fb8e3c3c40c None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:19.273587 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 614/1233] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:38:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:38:19.279408 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84544a83-eef0-4a36-bcab-196dd4b37ac7 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:19.289238 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84544a83-eef0-4a36-bcab-196dd4b37ac7 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:19.289238 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84544a83-eef0-4a36-bcab-196dd4b37ac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:19.298852 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-84544a83-eef0-4a36-bcab-196dd4b37ac7 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:38:19.300051 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-84544a83-eef0-4a36-bcab-196dd4b37ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:38:19.374053 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 620/1234] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:38:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:38:19.643622 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98d3122-e2c3-4ac0-acaf-3fb8e3c3c40c None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:19.670804 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:38:19.751657 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98d3122-e2c3-4ac0-acaf-3fb8e3c3c40c None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:19.788386 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea8dfc9-0cf3-4e51-b6d5-75adcd9178cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:19.789175 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea8dfc9-0cf3-4e51-b6d5-75adcd9178cc None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:19.789538 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea8dfc9-0cf3-4e51-b6d5-75adcd9178cc None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:19.789895 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea8dfc9-0cf3-4e51-b6d5-75adcd9178cc None admin] PATH_INFO: `/v3/roles/service` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:19.805451 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea8dfc9-0cf3-4e51-b6d5-75adcd9178cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:38:19.810396 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-8ea8dfc9-0cf3-4e51-b6d5-75adcd9178cc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:38:19.810396 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:38:19.810396 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:38:19.810396 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:38:19.810396 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:38:19.810396 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:38:19.810396 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:38:19.810396 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:38:19.810396 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:38:19.810396 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:38:19.810396 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:38:19.810396 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:38:19.810396 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:38:19.810396 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:38:19.810396 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:38:19.810396 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:38:19.812322 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:38:19.812322 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:38:19.812322 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:38:19.812322 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:38:19.812322 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:38:19.812322 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:38:19.812322 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:38:19.812322 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:38:19.812322 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:38:19.812322 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:38:19.812322 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:38:19.812322 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:38:19.812322 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:38:19.812322 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:38:19.812322 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:38:19.812322 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:38:19.813524 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:38:19.813524 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:38:19.813524 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:38:19.813524 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:38:19.813524 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:38:19.813524 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:38:19.813524 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:38:19.813524 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:38:19.813524 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:38:19.813524 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:38:19.813524 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:38:19.813524 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:38:19.817005 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 615/1235] 10.222.0.55 () {56 vars in 1102 bytes} [Wed Aug 24 08:38:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:38:19.820219 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84544a83-eef0-4a36-bcab-196dd4b37ac7 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:19.872930 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:38:19.927762 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84544a83-eef0-4a36-bcab-196dd4b37ac7 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:19.964531 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bd4cef-d5d2-4c27-90b8-378cb47f0145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:19.965194 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bd4cef-d5d2-4c27-90b8-378cb47f0145 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:19.965467 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bd4cef-d5d2-4c27-90b8-378cb47f0145 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:19.965741 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bd4cef-d5d2-4c27-90b8-378cb47f0145 None admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:19.966403 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bd4cef-d5d2-4c27-90b8-378cb47f0145 None admin] RBAC: Adding query filter params (name=service) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:38:19.966697 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bd4cef-d5d2-4c27-90b8-378cb47f0145 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:38:19.968293 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bd4cef-d5d2-4c27-90b8-378cb47f0145 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:38:19.977424 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 621/1236] 10.222.0.55 () {56 vars in 1103 bytes} [Wed Aug 24 08:38:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:38:19.984213 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea8dfc9-0cf3-4e51-b6d5-75adcd9178cc None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:20.087702 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:38:20.106931 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea8dfc9-0cf3-4e51-b6d5-75adcd9178cc None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:20.144181 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ce38b4-650a-46e1-bb79-13f946aecc70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:20.144852 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ce38b4-650a-46e1-bb79-13f946aecc70 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:20.145127 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ce38b4-650a-46e1-bb79-13f946aecc70 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:20.145393 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ce38b4-650a-46e1-bb79-13f946aecc70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:20.156933 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ce38b4-650a-46e1-bb79-13f946aecc70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:38:20.158396 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ce38b4-650a-46e1-bb79-13f946aecc70 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:38:20.173688 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-56ce38b4-650a-46e1-bb79-13f946aecc70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:38:20.173688 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:38:20.173688 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:38:20.173688 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:38:20.173688 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:38:20.173688 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:38:20.173688 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:38:20.173688 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:38:20.173688 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:38:20.173688 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:38:20.173688 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:38:20.173688 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:38:20.173688 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:38:20.173688 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:38:20.173688 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:38:20.173688 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:38:20.174945 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:38:20.174945 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:38:20.174945 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:38:20.174945 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:38:20.174945 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:38:20.174945 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:38:20.174945 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:38:20.174945 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:38:20.174945 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:38:20.174945 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:38:20.174945 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:38:20.174945 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:38:20.174945 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:38:20.174945 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:38:20.174945 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:38:20.174945 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:38:20.174945 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:38:20.176271 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:38:20.176271 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:38:20.176271 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:38:20.176271 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 616/1237] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:38:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:38:20.181556 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bd4cef-d5d2-4c27-90b8-378cb47f0145 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:20.292871 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bd4cef-d5d2-4c27-90b8-378cb47f0145 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:20.302126 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:38:20.327390 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8225311-145e-49d7-baa0-bdf48764a7c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:20.328500 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8225311-145e-49d7-baa0-bdf48764a7c2 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:20.328808 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8225311-145e-49d7-baa0-bdf48764a7c2 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:20.329076 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8225311-145e-49d7-baa0-bdf48764a7c2 None admin] PATH_INFO: `/v3/domains` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:20.329725 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8225311-145e-49d7-baa0-bdf48764a7c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:38:20.330025 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8225311-145e-49d7-baa0-bdf48764a7c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:38:20.331391 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8225311-145e-49d7-baa0-bdf48764a7c2 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:38:20.344415 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 622/1238] 10.222.0.55 () {56 vars in 1109 bytes} [Wed Aug 24 08:38:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:38:20.350470 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ce38b4-650a-46e1-bb79-13f946aecc70 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:20.454859 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ce38b4-650a-46e1-bb79-13f946aecc70 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:20.488452 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e2f820-1d8e-4db0-8eb3-83a7413cc501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:20.489106 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e2f820-1d8e-4db0-8eb3-83a7413cc501 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:20.489384 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e2f820-1d8e-4db0-8eb3-83a7413cc501 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:20.489648 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e2f820-1d8e-4db0-8eb3-83a7413cc501 None admin] PATH_INFO: `/v3/users/placement` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:20.492336 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:38:20.497575 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e2f820-1d8e-4db0-8eb3-83a7413cc501 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:38:20.693153 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e2f820-1d8e-4db0-8eb3-83a7413cc501 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:38:20.693153 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-58e2f820-1d8e-4db0-8eb3-83a7413cc501 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 08:38:20.693153 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:38:20.693153 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:38:20.693153 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:38:20.693153 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:38:20.693153 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:38:20.693153 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:38:20.693153 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:38:20.693153 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:38:20.693153 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:38:20.693153 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:38:20.693153 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:38:20.693153 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:38:20.694192 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:38:20.694192 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:38:20.694192 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:38:20.694192 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:38:20.694192 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:38:20.694192 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:38:20.694192 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:38:20.694192 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:38:20.694192 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:38:20.694192 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:38:20.694192 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:38:20.694192 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:38:20.694192 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:38:20.694192 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:38:20.694192 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:38:20.694192 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:38:20.694192 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:38:20.694192 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:38:20.695437 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:38:20.695437 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:38:20.695437 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:38:20.695437 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:38:20.695437 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:38:20.695437 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:38:20.695437 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:38:20.695437 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:38:20.695437 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:38:20.695437 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:38:20.695437 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:38:20.695437 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 08:38:20.695437 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:38:20.695437 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 617/1239] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:38:20 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:38:20.696382 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8225311-145e-49d7-baa0-bdf48764a7c2 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:20.696382 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8225311-145e-49d7-baa0-bdf48764a7c2 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:20.696382 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e1a4af-5f44-4905-9bde-28e799da6e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:20.696382 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e1a4af-5f44-4905-9bde-28e799da6e66 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:20.696772 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e1a4af-5f44-4905-9bde-28e799da6e66 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:20.696772 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e1a4af-5f44-4905-9bde-28e799da6e66 None admin] PATH_INFO: `/v3/users` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:20.696772 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e1a4af-5f44-4905-9bde-28e799da6e66 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:38:20.696772 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e1a4af-5f44-4905-9bde-28e799da6e66 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:38:20.696772 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e1a4af-5f44-4905-9bde-28e799da6e66 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:38:20.696772 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 623/1240] 10.222.0.55 () {56 vars in 1143 bytes} [Wed Aug 24 08:38:20 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:38:20.697219 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e2f820-1d8e-4db0-8eb3-83a7413cc501 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:20.783919 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e2f820-1d8e-4db0-8eb3-83a7413cc501 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:20.818583 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ab9012-0e0d-4854-8609-33e3a45f312c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:20.819229 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ab9012-0e0d-4854-8609-33e3a45f312c None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:20.819510 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ab9012-0e0d-4854-8609-33e3a45f312c None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:20.819805 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ab9012-0e0d-4854-8609-33e3a45f312c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:20.889745 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ab9012-0e0d-4854-8609-33e3a45f312c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:38:20.891546 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ab9012-0e0d-4854-8609-33e3a45f312c None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:38:20.995830 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 618/1241] 10.222.0.55 () {58 vars in 1282 bytes} [Wed Aug 24 08:38:20 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 08:38:21.001727 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e1a4af-5f44-4905-9bde-28e799da6e66 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:21.028731 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:38:21.080740 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:38:21.100769 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e1a4af-5f44-4905-9bde-28e799da6e66 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:21.134430 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea01dd1-5b9c-434b-947a-40db7f5f9dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:21.135063 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea01dd1-5b9c-434b-947a-40db7f5f9dba None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:21.135344 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea01dd1-5b9c-434b-947a-40db7f5f9dba None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:21.135614 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea01dd1-5b9c-434b-947a-40db7f5f9dba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:21.146176 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea01dd1-5b9c-434b-947a-40db7f5f9dba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:38:21.147609 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea01dd1-5b9c-434b-947a-40db7f5f9dba None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:38:21.162259 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-0ea01dd1-5b9c-434b-947a-40db7f5f9dba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:38:21.162259 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:38:21.162259 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:38:21.162259 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:38:21.162259 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:38:21.162259 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:38:21.162259 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:38:21.162259 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:38:21.162259 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:38:21.162259 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:38:21.162259 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:38:21.162259 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:38:21.162259 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:38:21.162259 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:38:21.162259 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:38:21.162259 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:38:21.163392 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:38:21.163392 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:38:21.163392 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:38:21.163392 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:38:21.163392 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:38:21.163392 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:38:21.163392 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:38:21.163392 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:38:21.163392 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:38:21.163392 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:38:21.163392 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:38:21.163392 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:38:21.163392 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:38:21.163392 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:38:21.163392 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:38:21.163392 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:38:21.163392 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:38:21.164722 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:38:21.164722 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:38:21.164722 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:38:21.164722 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 624/1242] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:38:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:38:21.169199 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ab9012-0e0d-4854-8609-33e3a45f312c None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:21.269668 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ab9012-0e0d-4854-8609-33e3a45f312c None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:21.303153 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96cfa25-48ff-4835-a420-8df5a9a69194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:21.303771 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96cfa25-48ff-4835-a420-8df5a9a69194 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:21.304091 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96cfa25-48ff-4835-a420-8df5a9a69194 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:21.304363 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96cfa25-48ff-4835-a420-8df5a9a69194 None admin] PATH_INFO: `/v3/domains` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:21.305030 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96cfa25-48ff-4835-a420-8df5a9a69194 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:38:21.305332 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96cfa25-48ff-4835-a420-8df5a9a69194 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:38:21.306680 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96cfa25-48ff-4835-a420-8df5a9a69194 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:38:21.318530 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 619/1243] 10.222.0.55 () {56 vars in 1109 bytes} [Wed Aug 24 08:38:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:38:21.324170 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea01dd1-5b9c-434b-947a-40db7f5f9dba None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:21.380430 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:38:21.402974 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:38:21.423797 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea01dd1-5b9c-434b-947a-40db7f5f9dba None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:21.456098 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd52cb7-2a74-4752-b909-1b2ccd19f511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:21.456743 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd52cb7-2a74-4752-b909-1b2ccd19f511 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:21.457011 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd52cb7-2a74-4752-b909-1b2ccd19f511 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:21.457278 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd52cb7-2a74-4752-b909-1b2ccd19f511 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:21.463733 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd52cb7-2a74-4752-b909-1b2ccd19f511 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:38:21.465183 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd52cb7-2a74-4752-b909-1b2ccd19f511 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:38:21.476716 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-2cd52cb7-2a74-4752-b909-1b2ccd19f511 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:38:21.476716 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:38:21.476716 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:38:21.476716 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:38:21.476716 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:38:21.476716 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:38:21.476716 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:38:21.476716 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:38:21.476716 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:38:21.476716 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:38:21.476716 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:38:21.476716 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:38:21.476716 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:38:21.476716 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:38:21.476716 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:38:21.477921 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:38:21.477921 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:38:21.477921 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:38:21.477921 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:38:21.477921 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:38:21.477921 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:38:21.477921 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:38:21.477921 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:38:21.477921 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:38:21.477921 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:38:21.477921 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:38:21.477921 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:38:21.477921 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:38:21.477921 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:38:21.477921 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:38:21.477921 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:38:21.477921 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:38:21.479048 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:38:21.479048 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:38:21.479048 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:38:21.479048 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:38:21.479048 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:38:21.479048 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:38:21.479048 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:38:21.479048 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:38:21.479048 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 625/1244] 10.222.0.55 () {56 vars in 1111 bytes} [Wed Aug 24 08:38:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:38:21.484444 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96cfa25-48ff-4835-a420-8df5a9a69194 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:21.582995 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96cfa25-48ff-4835-a420-8df5a9a69194 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:21.617494 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0371aa1b-ea82-4bb3-8434-9117851fca9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:21.618102 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0371aa1b-ea82-4bb3-8434-9117851fca9f None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:21.618376 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0371aa1b-ea82-4bb3-8434-9117851fca9f None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:21.618645 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0371aa1b-ea82-4bb3-8434-9117851fca9f None admin] PATH_INFO: `/v3/projects` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:21.619333 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0371aa1b-ea82-4bb3-8434-9117851fca9f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:38:21.619633 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0371aa1b-ea82-4bb3-8434-9117851fca9f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:38:21.621084 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0371aa1b-ea82-4bb3-8434-9117851fca9f None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:38:21.633908 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 620/1245] 10.222.0.55 () {56 vars in 1148 bytes} [Wed Aug 24 08:38:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:38:21.640176 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd52cb7-2a74-4752-b909-1b2ccd19f511 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:21.744463 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd52cb7-2a74-4752-b909-1b2ccd19f511 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:21.778940 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d3a226-507b-40b1-a2fa-b3c870a37e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:21.779549 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d3a226-507b-40b1-a2fa-b3c870a37e06 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:21.779831 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d3a226-507b-40b1-a2fa-b3c870a37e06 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:21.780133 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d3a226-507b-40b1-a2fa-b3c870a37e06 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:21.781399 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d3a226-507b-40b1-a2fa-b3c870a37e06 None admin] RBAC: Adding query filter params (role.id=bab90e02eebc473e9b911c8660ed9b76, scope.project.id=420b18725f2342159646fb86438318fd, user.id=cf0c99771661413aaa0f237af6b394ab) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:38:21.781708 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d3a226-507b-40b1-a2fa-b3c870a37e06 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:38:21.783130 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d3a226-507b-40b1-a2fa-b3c870a37e06 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:38:21.789080 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 626/1246] 10.222.0.55 () {56 vars in 1374 bytes} [Wed Aug 24 08:38:21 2022] GET /identity/v3/role_assignments?user.id=cf0c99771661413aaa0f237af6b394ab&scope.project.id=420b18725f2342159646fb86438318fd&role.id=bab90e02eebc473e9b911c8660ed9b76 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:38:23.750518 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c930aafa-ec38-47dc-a252-bffbf84826ca None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:23.751123 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c930aafa-ec38-47dc-a252-bffbf84826ca None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:23.751123 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c930aafa-ec38-47dc-a252-bffbf84826ca None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:23.752389 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 621/1247] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:38:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:38:23.757732 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fd445b-769f-412a-a8fc-48552d7049cb None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:23.758158 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fd445b-769f-412a-a8fc-48552d7049cb None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:23.758299 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fd445b-769f-412a-a8fc-48552d7049cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:23.764667 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:38:23.777445 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-27fd445b-769f-412a-a8fc-48552d7049cb None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:38:23.785736 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-27fd445b-769f-412a-a8fc-48552d7049cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:38:23.845246 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 627/1248] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:38:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:38:24.111693 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c930aafa-ec38-47dc-a252-bffbf84826ca None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:24.130053 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:38:24.217124 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c930aafa-ec38-47dc-a252-bffbf84826ca None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:24.254405 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a7624c-eb8e-4a18-8aac-7d8eb0113fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:24.255057 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a7624c-eb8e-4a18-8aac-7d8eb0113fc2 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:24.255334 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a7624c-eb8e-4a18-8aac-7d8eb0113fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:24.255605 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a7624c-eb8e-4a18-8aac-7d8eb0113fc2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:24.267531 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a7624c-eb8e-4a18-8aac-7d8eb0113fc2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:38:24.272213 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-26a7624c-eb8e-4a18-8aac-7d8eb0113fc2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:38:24.272213 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:38:24.272213 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:38:24.272213 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:38:24.272213 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:38:24.272213 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:38:24.272213 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:38:24.272213 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:38:24.272213 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:38:24.272213 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:38:24.272213 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:38:24.272213 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:38:24.272213 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:38:24.272213 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:38:24.272213 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:38:24.272213 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:38:24.273367 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:38:24.273367 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:38:24.273367 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:38:24.273367 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:38:24.273367 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:38:24.273367 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:38:24.273367 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:38:24.273367 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:38:24.273367 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:38:24.273367 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:38:24.273367 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:38:24.273367 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:38:24.273367 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:38:24.273367 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:38:24.273367 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:38:24.273367 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:38:24.274635 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:38:24.274635 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:38:24.274635 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:38:24.274635 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:38:24.274635 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:38:24.274635 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:38:24.274635 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:38:24.274635 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:38:24.274635 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:38:24.274635 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:38:24.274635 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:38:24.274635 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:38:24.274635 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 622/1249] 10.222.0.55 () {56 vars in 1096 bytes} [Wed Aug 24 08:38:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:38:24.279478 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fd445b-769f-412a-a8fc-48552d7049cb None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:24.382769 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fd445b-769f-412a-a8fc-48552d7049cb None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:24.411589 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:38:24.417287 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a2a683-8a29-4049-aa42-f973856e6563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:24.417904 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a2a683-8a29-4049-aa42-f973856e6563 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:24.418193 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a2a683-8a29-4049-aa42-f973856e6563 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:24.418465 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a2a683-8a29-4049-aa42-f973856e6563 None admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:24.419131 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a2a683-8a29-4049-aa42-f973856e6563 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:38:24.419436 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a2a683-8a29-4049-aa42-f973856e6563 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:38:24.420912 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a2a683-8a29-4049-aa42-f973856e6563 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:38:24.429831 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 628/1250] 10.222.0.55 () {56 vars in 1099 bytes} [Wed Aug 24 08:38:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:38:24.435560 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a7624c-eb8e-4a18-8aac-7d8eb0113fc2 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:24.516071 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:38:24.538380 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a7624c-eb8e-4a18-8aac-7d8eb0113fc2 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:24.572990 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96562e56-c637-459f-ad2e-e21ee7cbdbd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:24.573619 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96562e56-c637-459f-ad2e-e21ee7cbdbd2 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:24.573894 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96562e56-c637-459f-ad2e-e21ee7cbdbd2 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:24.574162 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96562e56-c637-459f-ad2e-e21ee7cbdbd2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:24.584949 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96562e56-c637-459f-ad2e-e21ee7cbdbd2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:38:24.586426 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96562e56-c637-459f-ad2e-e21ee7cbdbd2 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:38:24.600964 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-96562e56-c637-459f-ad2e-e21ee7cbdbd2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:38:24.600964 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:38:24.600964 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:38:24.600964 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:38:24.600964 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:38:24.600964 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:38:24.600964 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:38:24.600964 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:38:24.600964 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:38:24.600964 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:38:24.600964 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:38:24.600964 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:38:24.600964 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:38:24.600964 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:38:24.600964 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:38:24.600964 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:38:24.611910 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:38:24.611910 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:38:24.611910 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:38:24.611910 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:38:24.611910 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:38:24.611910 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:38:24.611910 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:38:24.611910 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:38:24.611910 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:38:24.611910 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:38:24.611910 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:38:24.611910 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:38:24.611910 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:38:24.611910 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:38:24.611910 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:38:24.611910 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:38:24.611910 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:38:24.612887 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:38:24.612887 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:38:24.612887 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:38:24.612887 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 623/1251] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:38:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:38:24.613167 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a2a683-8a29-4049-aa42-f973856e6563 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:24.709687 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a2a683-8a29-4049-aa42-f973856e6563 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:24.742697 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d912592b-c3ac-4f15-b8b6-70305dd2f89d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:24.743295 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d912592b-c3ac-4f15-b8b6-70305dd2f89d None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:24.743579 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d912592b-c3ac-4f15-b8b6-70305dd2f89d None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:24.743847 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d912592b-c3ac-4f15-b8b6-70305dd2f89d None admin] PATH_INFO: `/v3/domains` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:24.744487 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d912592b-c3ac-4f15-b8b6-70305dd2f89d None admin] RBAC: Adding query filter params (name=Default) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:38:24.744814 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d912592b-c3ac-4f15-b8b6-70305dd2f89d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:38:24.746175 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d912592b-c3ac-4f15-b8b6-70305dd2f89d None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:38:24.758606 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 629/1252] 10.222.0.55 () {56 vars in 1109 bytes} [Wed Aug 24 08:38:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:38:24.764584 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96562e56-c637-459f-ad2e-e21ee7cbdbd2 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:24.866317 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96562e56-c637-459f-ad2e-e21ee7cbdbd2 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:24.892270 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:38:24.900077 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cab32b5-9e96-412b-be1b-24618a4fa128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:24.900760 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cab32b5-9e96-412b-be1b-24618a4fa128 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:24.901036 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cab32b5-9e96-412b-be1b-24618a4fa128 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:24.901301 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cab32b5-9e96-412b-be1b-24618a4fa128 None admin] PATH_INFO: `/v3/users/placement` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:24.909981 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cab32b5-9e96-412b-be1b-24618a4fa128 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:38:24.914063 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cab32b5-9e96-412b-be1b-24618a4fa128 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:38:24.923442 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-3cab32b5-9e96-412b-be1b-24618a4fa128 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 08:38:24.923442 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:38:24.923442 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:38:24.923442 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:38:24.923442 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:38:24.923442 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:38:24.923442 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:38:24.923442 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:38:24.923442 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:38:24.923442 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:38:24.923442 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:38:24.923442 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:38:24.923442 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:38:24.923442 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:38:24.923442 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:38:24.923442 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:38:24.924654 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:38:24.924654 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:38:24.924654 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:38:24.924654 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:38:24.924654 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:38:24.924654 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:38:24.924654 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:38:24.924654 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:38:24.924654 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:38:24.924654 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:38:24.924654 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:38:24.924654 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:38:24.924654 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:38:24.924654 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:38:24.924654 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:38:24.924654 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:38:24.924654 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:38:24.924654 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:38:24.925627 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:38:24.925627 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:38:24.925627 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:38:24.925627 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:38:24.925627 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:38:24.925627 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:38:24.925627 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:38:24.925627 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:38:24.925627 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 08:38:24.925627 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:38:24.925627 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 624/1253] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:38:24 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:38:24.930614 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d912592b-c3ac-4f15-b8b6-70305dd2f89d None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:24.987556 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:38:25.030172 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d912592b-c3ac-4f15-b8b6-70305dd2f89d None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:25.066499 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e63f1c5-95b5-4d0b-897a-46a01b6d506e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:25.067460 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e63f1c5-95b5-4d0b-897a-46a01b6d506e None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:25.067971 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e63f1c5-95b5-4d0b-897a-46a01b6d506e None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:25.068261 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e63f1c5-95b5-4d0b-897a-46a01b6d506e None admin] PATH_INFO: `/v3/users` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:25.069511 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e63f1c5-95b5-4d0b-897a-46a01b6d506e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:38:25.070246 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e63f1c5-95b5-4d0b-897a-46a01b6d506e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:38:25.072287 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e63f1c5-95b5-4d0b-897a-46a01b6d506e None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:38:25.091830 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 630/1254] 10.222.0.55 () {56 vars in 1143 bytes} [Wed Aug 24 08:38:24 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:38:25.098416 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cab32b5-9e96-412b-be1b-24618a4fa128 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:25.203107 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cab32b5-9e96-412b-be1b-24618a4fa128 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:25.240218 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d4c443-f490-4346-b931-22a94ed06a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:25.241027 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d4c443-f490-4346-b931-22a94ed06a34 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:25.241442 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d4c443-f490-4346-b931-22a94ed06a34 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:25.241837 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d4c443-f490-4346-b931-22a94ed06a34 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:25.302483 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:38:25.317671 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d4c443-f490-4346-b931-22a94ed06a34 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:38:25.319124 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d4c443-f490-4346-b931-22a94ed06a34 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:38:25.421922 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 625/1255] 10.222.0.55 () {58 vars in 1282 bytes} [Wed Aug 24 08:38:25 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 08:38:25.427677 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e63f1c5-95b5-4d0b-897a-46a01b6d506e None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:25.534631 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e63f1c5-95b5-4d0b-897a-46a01b6d506e None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:25.547572 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:38:25.568348 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f966a55f-70a5-4e62-b94b-1467d76c8eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:25.569004 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f966a55f-70a5-4e62-b94b-1467d76c8eb9 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:25.569278 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f966a55f-70a5-4e62-b94b-1467d76c8eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:25.569574 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f966a55f-70a5-4e62-b94b-1467d76c8eb9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:25.579330 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f966a55f-70a5-4e62-b94b-1467d76c8eb9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:38:25.580833 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f966a55f-70a5-4e62-b94b-1467d76c8eb9 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:38:25.595808 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-f966a55f-70a5-4e62-b94b-1467d76c8eb9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:38:25.595808 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:38:25.595808 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:38:25.595808 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:38:25.595808 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:38:25.595808 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:38:25.595808 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:38:25.595808 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:38:25.595808 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:38:25.595808 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:38:25.595808 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:38:25.595808 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:38:25.595808 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:38:25.595808 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:38:25.595808 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:38:25.595808 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:38:25.597181 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:38:25.597181 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:38:25.597181 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:38:25.597181 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:38:25.597181 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:38:25.597181 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:38:25.597181 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:38:25.597181 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:38:25.597181 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:38:25.597181 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:38:25.597181 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:38:25.597181 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:38:25.597181 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:38:25.597181 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:38:25.597181 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:38:25.597181 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:38:25.597181 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:38:25.598566 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:38:25.598566 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:38:25.598566 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:38:25.598566 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 631/1256] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:38:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:38:25.603042 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d4c443-f490-4346-b931-22a94ed06a34 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:25.695565 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:38:25.707975 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d4c443-f490-4346-b931-22a94ed06a34 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:25.740697 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af18bd4-a57c-48d7-a4a1-f23fb7336f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:25.741294 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af18bd4-a57c-48d7-a4a1-f23fb7336f9f None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:25.741595 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af18bd4-a57c-48d7-a4a1-f23fb7336f9f None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:25.741866 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af18bd4-a57c-48d7-a4a1-f23fb7336f9f None admin] PATH_INFO: `/v3/domains` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:25.742500 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af18bd4-a57c-48d7-a4a1-f23fb7336f9f None admin] RBAC: Adding query filter params (name=Default) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:38:25.742801 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af18bd4-a57c-48d7-a4a1-f23fb7336f9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:38:25.744160 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af18bd4-a57c-48d7-a4a1-f23fb7336f9f None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:38:25.755836 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 626/1257] 10.222.0.55 () {56 vars in 1109 bytes} [Wed Aug 24 08:38:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:38:25.761858 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f966a55f-70a5-4e62-b94b-1467d76c8eb9 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:25.865318 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f966a55f-70a5-4e62-b94b-1467d76c8eb9 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:25.900456 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da30ded-65a6-4469-bede-a7f1557dd283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:25.901117 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da30ded-65a6-4469-bede-a7f1557dd283 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:25.901394 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da30ded-65a6-4469-bede-a7f1557dd283 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:25.901664 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da30ded-65a6-4469-bede-a7f1557dd283 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:25.908621 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da30ded-65a6-4469-bede-a7f1557dd283 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:38:25.910057 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da30ded-65a6-4469-bede-a7f1557dd283 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:38:25.921629 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-7da30ded-65a6-4469-bede-a7f1557dd283 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:38:25.921629 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:38:25.921629 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:38:25.921629 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:38:25.921629 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:38:25.921629 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:38:25.921629 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:38:25.921629 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:38:25.921629 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:38:25.921629 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:38:25.921629 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:38:25.921629 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:38:25.921629 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:38:25.921629 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:38:25.921629 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:38:25.928808 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:38:25.928808 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:38:25.928808 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:38:25.928808 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:38:25.928808 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:38:25.928808 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:38:25.928808 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:38:25.928808 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:38:25.928808 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:38:25.928808 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:38:25.928808 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:38:25.928808 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:38:25.928808 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:38:25.928808 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:38:25.928808 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:38:25.928808 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:38:25.928808 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:38:25.929937 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:38:25.929937 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:38:25.929937 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:38:25.929937 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:38:25.929937 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:38:25.929937 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:38:25.929937 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:38:25.929937 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:38:25.929937 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 632/1258] 10.222.0.55 () {56 vars in 1111 bytes} [Wed Aug 24 08:38:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:38:25.930538 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af18bd4-a57c-48d7-a4a1-f23fb7336f9f None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:26.031633 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af18bd4-a57c-48d7-a4a1-f23fb7336f9f None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:26.065083 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3bc10a-1768-469c-aed5-a379c94e7e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:26.065875 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3bc10a-1768-469c-aed5-a379c94e7e08 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:26.066147 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3bc10a-1768-469c-aed5-a379c94e7e08 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:26.066415 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3bc10a-1768-469c-aed5-a379c94e7e08 None admin] PATH_INFO: `/v3/projects` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:26.067092 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3bc10a-1768-469c-aed5-a379c94e7e08 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:38:26.067391 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3bc10a-1768-469c-aed5-a379c94e7e08 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:38:26.068826 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3bc10a-1768-469c-aed5-a379c94e7e08 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:38:26.081129 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 627/1259] 10.222.0.55 () {56 vars in 1148 bytes} [Wed Aug 24 08:38:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:38:26.087411 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da30ded-65a6-4469-bede-a7f1557dd283 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:26.114151 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:38:26.192285 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da30ded-65a6-4469-bede-a7f1557dd283 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:26.227987 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc106166-3305-4fb7-a72e-f0d049d71db8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:26.228580 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc106166-3305-4fb7-a72e-f0d049d71db8 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:26.228860 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc106166-3305-4fb7-a72e-f0d049d71db8 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:26.229129 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc106166-3305-4fb7-a72e-f0d049d71db8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:26.229843 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc106166-3305-4fb7-a72e-f0d049d71db8 None admin] RBAC: Adding query filter params (role.id=b870cd1acaa744e1ae1d58b9ac283821, scope.project.id=420b18725f2342159646fb86438318fd, user.id=cf0c99771661413aaa0f237af6b394ab) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:38:26.230142 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc106166-3305-4fb7-a72e-f0d049d71db8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:38:26.231553 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc106166-3305-4fb7-a72e-f0d049d71db8 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:38:26.237113 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 633/1260] 10.222.0.55 () {56 vars in 1374 bytes} [Wed Aug 24 08:38:26 2022] GET /identity/v3/role_assignments?user.id=cf0c99771661413aaa0f237af6b394ab&scope.project.id=420b18725f2342159646fb86438318fd&role.id=b870cd1acaa744e1ae1d58b9ac283821 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:38:28.122195 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea0ff03-ecdc-454b-9695-b854cc0a8c56 None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:28.122759 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea0ff03-ecdc-454b-9695-b854cc0a8c56 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:28.122759 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea0ff03-ecdc-454b-9695-b854cc0a8c56 None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:28.124120 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 628/1261] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:38:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:38:28.130185 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0525b286-2548-4d19-9cb9-c62ed067f199 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:28.130583 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0525b286-2548-4d19-9cb9-c62ed067f199 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:28.130817 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0525b286-2548-4d19-9cb9-c62ed067f199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:28.148971 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-0525b286-2548-4d19-9cb9-c62ed067f199 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:38:28.150161 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-0525b286-2548-4d19-9cb9-c62ed067f199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:38:28.221420 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 634/1262] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:38:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:38:28.495488 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea0ff03-ecdc-454b-9695-b854cc0a8c56 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:28.512081 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:38:28.603064 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea0ff03-ecdc-454b-9695-b854cc0a8c56 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:28.638247 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933f8410-3590-4be3-a55e-1978f698e6e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:28.638876 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933f8410-3590-4be3-a55e-1978f698e6e1 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:28.639155 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933f8410-3590-4be3-a55e-1978f698e6e1 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:28.639427 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933f8410-3590-4be3-a55e-1978f698e6e1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:28.650029 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933f8410-3590-4be3-a55e-1978f698e6e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:38:28.654626 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-933f8410-3590-4be3-a55e-1978f698e6e1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:38:28.654626 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:38:28.654626 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:38:28.654626 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:38:28.654626 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:38:28.654626 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:38:28.654626 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:38:28.654626 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:38:28.654626 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:38:28.654626 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:38:28.654626 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:38:28.654626 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:38:28.654626 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:38:28.654626 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:38:28.654626 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:38:28.654626 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:38:28.655794 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:38:28.655794 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:38:28.655794 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:38:28.655794 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:38:28.655794 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:38:28.655794 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:38:28.655794 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:38:28.655794 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:38:28.655794 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:38:28.655794 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:38:28.655794 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:38:28.655794 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:38:28.655794 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:38:28.655794 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:38:28.655794 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:38:28.655794 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:38:28.656750 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:38:28.656750 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:38:28.656750 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:38:28.656750 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:38:28.656750 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:38:28.656750 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:38:28.656750 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:38:28.656750 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:38:28.656750 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:38:28.656750 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:38:28.656750 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:38:28.656750 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:38:28.656750 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 629/1263] 10.222.0.55 () {56 vars in 1096 bytes} [Wed Aug 24 08:38:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:38:28.661845 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0525b286-2548-4d19-9cb9-c62ed067f199 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:28.680109 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:38:28.765870 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0525b286-2548-4d19-9cb9-c62ed067f199 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:28.799612 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a90222-f1eb-4874-8262-0cbe4a4d2f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:28.800208 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a90222-f1eb-4874-8262-0cbe4a4d2f0f None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:28.800530 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a90222-f1eb-4874-8262-0cbe4a4d2f0f None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:28.800783 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a90222-f1eb-4874-8262-0cbe4a4d2f0f None admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:28.801431 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a90222-f1eb-4874-8262-0cbe4a4d2f0f None admin] RBAC: Adding query filter params (name=admin) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:38:28.801732 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a90222-f1eb-4874-8262-0cbe4a4d2f0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:38:28.803079 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a90222-f1eb-4874-8262-0cbe4a4d2f0f None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:38:28.810800 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 635/1264] 10.222.0.55 () {56 vars in 1099 bytes} [Wed Aug 24 08:38:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:38:28.817422 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933f8410-3590-4be3-a55e-1978f698e6e1 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:28.884095 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:38:28.923788 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933f8410-3590-4be3-a55e-1978f698e6e1 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:28.961240 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03afbe7a-3044-48c1-9db7-333d0b38dca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:28.961914 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03afbe7a-3044-48c1-9db7-333d0b38dca5 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:28.962193 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03afbe7a-3044-48c1-9db7-333d0b38dca5 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:28.962465 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03afbe7a-3044-48c1-9db7-333d0b38dca5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:28.974892 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03afbe7a-3044-48c1-9db7-333d0b38dca5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:38:28.976427 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03afbe7a-3044-48c1-9db7-333d0b38dca5 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:38:28.992510 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-03afbe7a-3044-48c1-9db7-333d0b38dca5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:38:28.992510 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:38:28.992510 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:38:28.992510 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:38:28.992510 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:38:28.992510 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:38:28.992510 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:38:28.992510 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:38:28.992510 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:38:28.992510 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:38:28.992510 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:38:28.992510 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:38:28.992510 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:38:28.992510 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:38:28.992510 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:38:28.992510 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:38:28.993729 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:38:28.993729 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:38:28.993729 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:38:28.993729 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:38:28.993729 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:38:28.993729 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:38:28.993729 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:38:28.993729 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:38:28.993729 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:38:28.993729 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:38:28.993729 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:38:28.993729 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:38:28.993729 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:38:28.993729 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:38:28.993729 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:38:28.993729 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:38:28.993729 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:38:28.994860 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:38:28.994860 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:38:28.994860 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:38:28.994860 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 630/1265] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:38:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:38:29.000384 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a90222-f1eb-4874-8262-0cbe4a4d2f0f None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:29.082481 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:38:29.101286 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a90222-f1eb-4874-8262-0cbe4a4d2f0f None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:29.135675 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bdb72d-7a4e-49fa-876c-8c2b1c31bf45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:29.136291 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bdb72d-7a4e-49fa-876c-8c2b1c31bf45 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:29.136568 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bdb72d-7a4e-49fa-876c-8c2b1c31bf45 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:29.136847 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bdb72d-7a4e-49fa-876c-8c2b1c31bf45 None admin] PATH_INFO: `/v3/domains` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:29.137482 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bdb72d-7a4e-49fa-876c-8c2b1c31bf45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:38:29.137784 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bdb72d-7a4e-49fa-876c-8c2b1c31bf45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:38:29.139139 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bdb72d-7a4e-49fa-876c-8c2b1c31bf45 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:38:29.150927 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 636/1266] 10.222.0.55 () {56 vars in 1109 bytes} [Wed Aug 24 08:38:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:38:29.158499 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03afbe7a-3044-48c1-9db7-333d0b38dca5 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:29.262528 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03afbe7a-3044-48c1-9db7-333d0b38dca5 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:29.286054 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:38:29.299450 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa0c76c-8968-4267-8dae-2c09c497ba5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:29.300077 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa0c76c-8968-4267-8dae-2c09c497ba5e None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:29.300354 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa0c76c-8968-4267-8dae-2c09c497ba5e None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:29.300650 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa0c76c-8968-4267-8dae-2c09c497ba5e None admin] PATH_INFO: `/v3/users/placement` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:29.307846 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa0c76c-8968-4267-8dae-2c09c497ba5e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:38:29.309347 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa0c76c-8968-4267-8dae-2c09c497ba5e None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:38:29.320806 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-aaa0c76c-8968-4267-8dae-2c09c497ba5e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 08:38:29.320806 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:38:29.320806 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:38:29.320806 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:38:29.320806 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:38:29.320806 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:38:29.320806 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:38:29.320806 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:38:29.320806 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:38:29.320806 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:38:29.320806 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:38:29.320806 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:38:29.320806 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:38:29.320806 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:38:29.320806 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:38:29.320806 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:38:29.324055 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:38:29.324055 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:38:29.324055 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:38:29.324055 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:38:29.324055 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:38:29.324055 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:38:29.324055 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:38:29.324055 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:38:29.324055 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:38:29.324055 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:38:29.324055 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:38:29.324055 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:38:29.324055 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:38:29.324055 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:38:29.324055 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:38:29.324055 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:38:29.324055 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:38:29.324055 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:38:29.325152 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:38:29.325152 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:38:29.325152 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:38:29.325152 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:38:29.325152 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:38:29.325152 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:38:29.325152 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:38:29.325152 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:38:29.325152 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 08:38:29.325152 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:38:29.325152 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 631/1267] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:38:29 2022] GET /identity/v3/users/placement => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:38:29.328566 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bdb72d-7a4e-49fa-876c-8c2b1c31bf45 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:29.427310 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bdb72d-7a4e-49fa-876c-8c2b1c31bf45 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:29.461285 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6e96b3-4e85-4ca9-907b-4e6375fa33d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:29.461879 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6e96b3-4e85-4ca9-907b-4e6375fa33d1 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:29.462152 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6e96b3-4e85-4ca9-907b-4e6375fa33d1 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:29.462420 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6e96b3-4e85-4ca9-907b-4e6375fa33d1 None admin] PATH_INFO: `/v3/users` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:29.463135 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6e96b3-4e85-4ca9-907b-4e6375fa33d1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:38:29.463433 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6e96b3-4e85-4ca9-907b-4e6375fa33d1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:38:29.464871 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6e96b3-4e85-4ca9-907b-4e6375fa33d1 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:38:29.473386 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:38:29.481771 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 637/1268] 10.222.0.55 () {56 vars in 1143 bytes} [Wed Aug 24 08:38:29 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:38:29.487571 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa0c76c-8968-4267-8dae-2c09c497ba5e None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:29.585144 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa0c76c-8968-4267-8dae-2c09c497ba5e None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:29.618079 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad91d117-63c0-46f8-ad31-cc662dbacf75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:29.618767 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad91d117-63c0-46f8-ad31-cc662dbacf75 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:29.619039 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad91d117-63c0-46f8-ad31-cc662dbacf75 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:29.619304 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad91d117-63c0-46f8-ad31-cc662dbacf75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:29.630186 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad91d117-63c0-46f8-ad31-cc662dbacf75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:38:29.663720 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad91d117-63c0-46f8-ad31-cc662dbacf75 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:38:29.663720 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-ad91d117-63c0-46f8-ad31-cc662dbacf75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:38:29.663720 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:38:29.663720 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:38:29.663720 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:38:29.663720 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:38:29.663720 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:38:29.663720 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:38:29.663720 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:38:29.663720 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:38:29.663720 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:38:29.663720 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:38:29.663720 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:38:29.663720 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:38:29.664741 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:38:29.664741 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:38:29.664741 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:38:29.664741 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:38:29.664741 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:38:29.664741 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:38:29.664741 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:38:29.664741 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:38:29.664741 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:38:29.664741 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:38:29.664741 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:38:29.664741 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:38:29.664741 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:38:29.664741 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:38:29.664741 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:38:29.664741 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:38:29.664741 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:38:29.665675 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:38:29.665675 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:38:29.665675 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:38:29.665675 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:38:29.665675 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:38:29.665675 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:38:29.665675 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 632/1269] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:38:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:38:29.666081 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6e96b3-4e85-4ca9-907b-4e6375fa33d1 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:29.749117 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6e96b3-4e85-4ca9-907b-4e6375fa33d1 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:29.783587 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718b00ac-71a7-416f-bd3f-6459147b1cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:29.784186 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718b00ac-71a7-416f-bd3f-6459147b1cf9 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:29.784467 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718b00ac-71a7-416f-bd3f-6459147b1cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:29.784769 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718b00ac-71a7-416f-bd3f-6459147b1cf9 None admin] PATH_INFO: `/v3/domains` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:29.785408 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718b00ac-71a7-416f-bd3f-6459147b1cf9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:38:29.785712 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718b00ac-71a7-416f-bd3f-6459147b1cf9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:38:29.787064 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718b00ac-71a7-416f-bd3f-6459147b1cf9 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:38:29.798706 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 638/1270] 10.222.0.55 () {56 vars in 1109 bytes} [Wed Aug 24 08:38:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:38:29.804404 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad91d117-63c0-46f8-ad31-cc662dbacf75 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:29.836748 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:38:29.906547 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad91d117-63c0-46f8-ad31-cc662dbacf75 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:29.941354 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cb3cc5-fc19-4044-b5f2-fa2867055386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:29.941981 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cb3cc5-fc19-4044-b5f2-fa2867055386 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:29.942259 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cb3cc5-fc19-4044-b5f2-fa2867055386 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:29.942528 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cb3cc5-fc19-4044-b5f2-fa2867055386 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:29.949159 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cb3cc5-fc19-4044-b5f2-fa2867055386 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:38:29.950738 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cb3cc5-fc19-4044-b5f2-fa2867055386 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:38:29.961757 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-d4cb3cc5-fc19-4044-b5f2-fa2867055386 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:38:29.961757 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:38:29.961757 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:38:29.961757 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:38:29.961757 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:38:29.961757 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:38:29.961757 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:38:29.961757 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:38:29.961757 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:38:29.961757 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:38:29.961757 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:38:29.961757 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:38:29.961757 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:38:29.961757 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:38:29.961757 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:38:29.963004 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:38:29.963004 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:38:29.963004 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:38:29.963004 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:38:29.963004 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:38:29.963004 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:38:29.963004 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:38:29.963004 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:38:29.963004 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:38:29.963004 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:38:29.963004 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:38:29.963004 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:38:29.963004 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:38:29.963004 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:38:29.963004 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:38:29.963004 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:38:29.963004 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:38:29.964410 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:38:29.964410 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:38:29.964410 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:38:29.964410 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:38:29.964410 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:38:29.964410 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:38:29.964410 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:38:29.964410 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:38:29.964410 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 633/1271] 10.222.0.55 () {56 vars in 1111 bytes} [Wed Aug 24 08:38:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:38:29.969468 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718b00ac-71a7-416f-bd3f-6459147b1cf9 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:30.029971 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:38:30.072856 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718b00ac-71a7-416f-bd3f-6459147b1cf9 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:30.106642 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055e411d-6c68-4709-b65b-a63082f822b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:30.107236 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055e411d-6c68-4709-b65b-a63082f822b9 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:30.107511 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055e411d-6c68-4709-b65b-a63082f822b9 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:30.107779 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055e411d-6c68-4709-b65b-a63082f822b9 None admin] PATH_INFO: `/v3/projects` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:30.109120 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055e411d-6c68-4709-b65b-a63082f822b9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:38:30.109431 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055e411d-6c68-4709-b65b-a63082f822b9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:38:30.110847 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055e411d-6c68-4709-b65b-a63082f822b9 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:38:30.122372 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 639/1272] 10.222.0.55 () {56 vars in 1148 bytes} [Wed Aug 24 08:38:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:38:30.128741 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cb3cc5-fc19-4044-b5f2-fa2867055386 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:30.226404 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:38:30.235592 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cb3cc5-fc19-4044-b5f2-fa2867055386 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:30.269699 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cf1c77-078e-418f-8303-2adca7b2ee82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:30.270349 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cf1c77-078e-418f-8303-2adca7b2ee82 None admin] REQUEST_METHOD: `PUT` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:30.270634 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cf1c77-078e-418f-8303-2adca7b2ee82 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:30.270907 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cf1c77-078e-418f-8303-2adca7b2ee82 None admin] PATH_INFO: `/v3/projects/420b18725f2342159646fb86438318fd/users/cf0c99771661413aaa0f237af6b394ab/roles/b870cd1acaa744e1ae1d58b9ac283821` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:30.272808 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cf1c77-078e-418f-8303-2adca7b2ee82 None admin] RBAC: Authorizing `identity:create_grant(project_id=420b18725f2342159646fb86438318fd, user_id=cf0c99771661413aaa0f237af6b394ab, role_id=b870cd1acaa744e1ae1d58b9ac283821)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:38:30.274327 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cf1c77-078e-418f-8303-2adca7b2ee82 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:38:30.482651 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 634/1273] 10.222.0.55 () {58 vars in 1439 bytes} [Wed Aug 24 08:38:30 2022] PUT /identity/v3/projects/420b18725f2342159646fb86438318fd/users/cf0c99771661413aaa0f237af6b394ab/roles/b870cd1acaa744e1ae1d58b9ac283821 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:38:32.365410 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293a9b21-f2b8-4651-8cb1-3a3eab270b56 None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:32.365988 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293a9b21-f2b8-4651-8cb1-3a3eab270b56 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:32.365988 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293a9b21-f2b8-4651-8cb1-3a3eab270b56 None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:32.367274 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 640/1274] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:38:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:38:32.372627 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a335feb-dda6-49ea-a976-82ad77b944c9 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:32.373051 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a335feb-dda6-49ea-a976-82ad77b944c9 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:32.373158 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a335feb-dda6-49ea-a976-82ad77b944c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:32.390991 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-6a335feb-dda6-49ea-a976-82ad77b944c9 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:38:32.392179 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-6a335feb-dda6-49ea-a976-82ad77b944c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:38:32.458673 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 635/1275] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:38:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:38:32.731913 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293a9b21-f2b8-4651-8cb1-3a3eab270b56 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:32.834277 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293a9b21-f2b8-4651-8cb1-3a3eab270b56 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:32.852011 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:38:32.868871 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b769d58-2078-44e8-9b5b-42ff50b701e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:32.869566 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b769d58-2078-44e8-9b5b-42ff50b701e0 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:32.875386 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b769d58-2078-44e8-9b5b-42ff50b701e0 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:32.875386 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b769d58-2078-44e8-9b5b-42ff50b701e0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:32.880563 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b769d58-2078-44e8-9b5b-42ff50b701e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:38:32.885361 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-9b769d58-2078-44e8-9b5b-42ff50b701e0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:38:32.885361 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:38:32.885361 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:38:32.885361 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:38:32.885361 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:38:32.885361 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:38:32.885361 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:38:32.885361 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:38:32.885361 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:38:32.885361 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:38:32.885361 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:38:32.885361 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:38:32.885361 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:38:32.885361 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:38:32.885361 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:38:32.885361 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:38:32.886499 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:38:32.886499 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:38:32.886499 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:38:32.886499 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:38:32.886499 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:38:32.886499 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:38:32.886499 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:38:32.886499 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:38:32.886499 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:38:32.886499 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:38:32.886499 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:38:32.886499 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:38:32.886499 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:38:32.886499 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:38:32.886499 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:38:32.886499 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:38:32.887662 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:38:32.887662 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:38:32.887662 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:38:32.887662 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:38:32.887662 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:38:32.887662 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:38:32.887662 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:38:32.887662 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:38:32.887662 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:38:32.887662 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:38:32.887662 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:38:32.887662 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:38:32.887662 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 641/1276] 10.222.0.55 () {56 vars in 1096 bytes} [Wed Aug 24 08:38:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:38:32.893470 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a335feb-dda6-49ea-a976-82ad77b944c9 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:32.975584 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:38:32.993184 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a335feb-dda6-49ea-a976-82ad77b944c9 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:33.026479 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7775a65b-3b6e-4258-bad9-19ce4d09a34d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:33.027076 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7775a65b-3b6e-4258-bad9-19ce4d09a34d None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:33.027349 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7775a65b-3b6e-4258-bad9-19ce4d09a34d None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:33.027617 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7775a65b-3b6e-4258-bad9-19ce4d09a34d None admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:33.028266 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7775a65b-3b6e-4258-bad9-19ce4d09a34d None admin] RBAC: Adding query filter params (name=admin) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:38:33.028580 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7775a65b-3b6e-4258-bad9-19ce4d09a34d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:38:33.030012 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7775a65b-3b6e-4258-bad9-19ce4d09a34d None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:38:33.037807 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 636/1277] 10.222.0.55 () {56 vars in 1099 bytes} [Wed Aug 24 08:38:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:38:33.043706 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b769d58-2078-44e8-9b5b-42ff50b701e0 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:33.149934 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b769d58-2078-44e8-9b5b-42ff50b701e0 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:33.185452 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12295670-103f-48ef-98a0-ce50817655d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:33.186157 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12295670-103f-48ef-98a0-ce50817655d7 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:33.186438 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12295670-103f-48ef-98a0-ce50817655d7 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:33.186709 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12295670-103f-48ef-98a0-ce50817655d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:33.201921 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12295670-103f-48ef-98a0-ce50817655d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:38:33.201921 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12295670-103f-48ef-98a0-ce50817655d7 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:38:33.214628 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-12295670-103f-48ef-98a0-ce50817655d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:38:33.214628 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:38:33.214628 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:38:33.214628 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:38:33.214628 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:38:33.214628 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:38:33.214628 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:38:33.214628 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:38:33.214628 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:38:33.214628 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:38:33.214628 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:38:33.214628 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:38:33.214628 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:38:33.214628 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:38:33.214628 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:38:33.214628 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:38:33.215752 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:38:33.215752 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:38:33.215752 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:38:33.215752 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:38:33.215752 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:38:33.215752 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:38:33.215752 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:38:33.215752 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:38:33.215752 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:38:33.215752 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:38:33.215752 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:38:33.215752 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:38:33.215752 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:38:33.215752 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:38:33.215752 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:38:33.215752 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:38:33.215752 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:38:33.217067 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:38:33.217067 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:38:33.217067 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:38:33.217067 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 642/1278] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:38:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:38:33.221704 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7775a65b-3b6e-4258-bad9-19ce4d09a34d None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:33.324044 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7775a65b-3b6e-4258-bad9-19ce4d09a34d None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:33.358314 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fccdbd-7a46-4b0b-823b-d9ebffaaf713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:33.359507 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fccdbd-7a46-4b0b-823b-d9ebffaaf713 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:33.359787 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fccdbd-7a46-4b0b-823b-d9ebffaaf713 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:33.360059 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fccdbd-7a46-4b0b-823b-d9ebffaaf713 None admin] PATH_INFO: `/v3/domains` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:33.360734 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fccdbd-7a46-4b0b-823b-d9ebffaaf713 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:38:33.361037 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fccdbd-7a46-4b0b-823b-d9ebffaaf713 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:38:33.362412 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fccdbd-7a46-4b0b-823b-d9ebffaaf713 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:38:33.375494 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 637/1279] 10.222.0.55 () {56 vars in 1109 bytes} [Wed Aug 24 08:38:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:38:33.381386 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12295670-103f-48ef-98a0-ce50817655d7 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:33.413194 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:38:33.484238 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12295670-103f-48ef-98a0-ce50817655d7 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:33.519966 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d1fe24-0872-4ac8-8dcb-556e61aba3a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:33.521795 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d1fe24-0872-4ac8-8dcb-556e61aba3a3 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:33.521795 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d1fe24-0872-4ac8-8dcb-556e61aba3a3 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:33.521795 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d1fe24-0872-4ac8-8dcb-556e61aba3a3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:33.528955 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d1fe24-0872-4ac8-8dcb-556e61aba3a3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:38:33.530423 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d1fe24-0872-4ac8-8dcb-556e61aba3a3 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:38:33.541549 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-64d1fe24-0872-4ac8-8dcb-556e61aba3a3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 08:38:33.541549 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:38:33.541549 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:38:33.541549 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:38:33.541549 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:38:33.541549 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:38:33.541549 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:38:33.541549 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:38:33.541549 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:38:33.541549 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:38:33.541549 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:38:33.541549 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:38:33.541549 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:38:33.541549 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:38:33.541549 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:38:33.541549 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:38:33.542496 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:38:33.542496 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:38:33.542496 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:38:33.542496 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:38:33.542496 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:38:33.542496 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:38:33.542496 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:38:33.542496 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:38:33.542496 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:38:33.542496 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:38:33.542496 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:38:33.542496 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:38:33.542496 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:38:33.542496 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:38:33.542496 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:38:33.542496 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:38:33.542496 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:38:33.542496 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:38:33.543539 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:38:33.543539 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:38:33.543539 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:38:33.543539 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:38:33.543539 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:38:33.543539 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:38:33.543539 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:38:33.543539 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:38:33.543539 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 08:38:33.543539 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:38:33.543539 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 643/1280] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:38:33 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:38:33.549200 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fccdbd-7a46-4b0b-823b-d9ebffaaf713 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:33.553260 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:38:33.646015 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fccdbd-7a46-4b0b-823b-d9ebffaaf713 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:33.679153 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ab76c7-d292-4bc6-83cd-c172d176ab47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:33.679746 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ab76c7-d292-4bc6-83cd-c172d176ab47 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:33.680019 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ab76c7-d292-4bc6-83cd-c172d176ab47 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:33.680283 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ab76c7-d292-4bc6-83cd-c172d176ab47 None admin] PATH_INFO: `/v3/users` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:33.681022 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ab76c7-d292-4bc6-83cd-c172d176ab47 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:38:33.681322 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ab76c7-d292-4bc6-83cd-c172d176ab47 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:38:33.682726 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ab76c7-d292-4bc6-83cd-c172d176ab47 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:38:33.700435 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 638/1281] 10.222.0.55 () {56 vars in 1143 bytes} [Wed Aug 24 08:38:33 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:38:33.706308 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d1fe24-0872-4ac8-8dcb-556e61aba3a3 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:33.799145 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:38:33.808392 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d1fe24-0872-4ac8-8dcb-556e61aba3a3 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:33.840956 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37b1ef8-d5cf-4685-a2f1-f754dc85daf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:33.841553 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37b1ef8-d5cf-4685-a2f1-f754dc85daf8 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:33.841872 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37b1ef8-d5cf-4685-a2f1-f754dc85daf8 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:33.842145 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37b1ef8-d5cf-4685-a2f1-f754dc85daf8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:33.908237 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37b1ef8-d5cf-4685-a2f1-f754dc85daf8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:38:33.909624 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37b1ef8-d5cf-4685-a2f1-f754dc85daf8 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:38:34.000828 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:38:34.010748 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 644/1282] 10.222.0.55 () {58 vars in 1282 bytes} [Wed Aug 24 08:38:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 08:38:34.017174 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ab76c7-d292-4bc6-83cd-c172d176ab47 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:34.078195 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:38:34.116419 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ab76c7-d292-4bc6-83cd-c172d176ab47 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:34.150332 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a440cada-3af9-4e40-9413-7da34dcc4391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:34.150955 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a440cada-3af9-4e40-9413-7da34dcc4391 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:34.151225 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a440cada-3af9-4e40-9413-7da34dcc4391 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:34.151491 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a440cada-3af9-4e40-9413-7da34dcc4391 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:34.161978 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a440cada-3af9-4e40-9413-7da34dcc4391 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:38:34.163355 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a440cada-3af9-4e40-9413-7da34dcc4391 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:38:34.177569 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-a440cada-3af9-4e40-9413-7da34dcc4391 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:38:34.177569 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:38:34.177569 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:38:34.177569 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:38:34.177569 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:38:34.177569 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:38:34.177569 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:38:34.177569 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:38:34.177569 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:38:34.177569 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:38:34.177569 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:38:34.177569 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:38:34.177569 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:38:34.177569 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:38:34.177569 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:38:34.177569 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:38:34.178716 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:38:34.178716 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:38:34.178716 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:38:34.178716 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:38:34.178716 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:38:34.178716 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:38:34.178716 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:38:34.178716 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:38:34.178716 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:38:34.178716 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:38:34.178716 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:38:34.178716 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:38:34.178716 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:38:34.178716 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:38:34.178716 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:38:34.178716 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:38:34.178716 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:38:34.179731 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:38:34.179731 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:38:34.179731 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:38:34.179731 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 639/1283] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:38:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:38:34.185332 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37b1ef8-d5cf-4685-a2f1-f754dc85daf8 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:34.287248 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37b1ef8-d5cf-4685-a2f1-f754dc85daf8 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:34.320050 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ddaaad-2932-4c80-bd15-8a24d841231e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:34.320725 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ddaaad-2932-4c80-bd15-8a24d841231e None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:34.321001 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ddaaad-2932-4c80-bd15-8a24d841231e None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:34.321279 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ddaaad-2932-4c80-bd15-8a24d841231e None admin] PATH_INFO: `/v3/domains` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:34.321910 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ddaaad-2932-4c80-bd15-8a24d841231e None admin] RBAC: Adding query filter params (name=Default) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:38:34.322210 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ddaaad-2932-4c80-bd15-8a24d841231e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:38:34.323555 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ddaaad-2932-4c80-bd15-8a24d841231e None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:38:34.335260 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 645/1284] 10.222.0.55 () {56 vars in 1109 bytes} [Wed Aug 24 08:38:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:38:34.340994 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a440cada-3af9-4e40-9413-7da34dcc4391 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:34.443538 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a440cada-3af9-4e40-9413-7da34dcc4391 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:34.456894 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:38:34.476826 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf9dc62-2729-4139-b5e5-68709b534914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:34.477448 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf9dc62-2729-4139-b5e5-68709b534914 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:34.477724 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf9dc62-2729-4139-b5e5-68709b534914 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:34.477993 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf9dc62-2729-4139-b5e5-68709b534914 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:34.485058 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf9dc62-2729-4139-b5e5-68709b534914 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:38:34.486483 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf9dc62-2729-4139-b5e5-68709b534914 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:38:34.497197 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application [None req-fbf9dc62-2729-4139-b5e5-68709b534914 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:38:34.497197 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:38:34.497197 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:38:34.497197 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:38:34.497197 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:38:34.497197 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:38:34.497197 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:38:34.497197 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:38:34.497197 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:38:34.497197 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:38:34.497197 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:38:34.497197 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:38:34.497197 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:38:34.497197 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:38:34.497197 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:38:34.498275 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:38:34.498275 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:38:34.498275 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:38:34.498275 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:38:34.498275 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:38:34.498275 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:38:34.498275 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:38:34.498275 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:38:34.498275 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application with Lock( Aug 24 08:38:34.498275 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:38:34.498275 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:38:34.498275 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:38:34.498275 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:38:34.498275 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:38:34.498275 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:38:34.498275 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:38:34.498275 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:38:34.499378 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:38:34.499378 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:38:34.499378 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:38:34.499378 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:38:34.499378 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:38:34.499378 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:38:34.499378 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:38:34.499378 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: ERROR keystone.server.flask.application Aug 24 08:38:34.499378 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 640/1285] 10.222.0.55 () {56 vars in 1111 bytes} [Wed Aug 24 08:38:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:38:34.504360 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ddaaad-2932-4c80-bd15-8a24d841231e None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:34.555003 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:38:34.608012 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ddaaad-2932-4c80-bd15-8a24d841231e None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:34.642273 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eadaef0-dc4a-4a52-916d-02efa7e89a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:34.642898 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eadaef0-dc4a-4a52-916d-02efa7e89a11 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:34.644218 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eadaef0-dc4a-4a52-916d-02efa7e89a11 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:34.644218 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eadaef0-dc4a-4a52-916d-02efa7e89a11 None admin] PATH_INFO: `/v3/projects` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:34.644218 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eadaef0-dc4a-4a52-916d-02efa7e89a11 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:38:34.644469 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eadaef0-dc4a-4a52-916d-02efa7e89a11 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:38:34.645906 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eadaef0-dc4a-4a52-916d-02efa7e89a11 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:38:34.658506 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 646/1286] 10.222.0.55 () {56 vars in 1148 bytes} [Wed Aug 24 08:38:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:38:34.664786 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf9dc62-2729-4139-b5e5-68709b534914 None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:34.763543 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf9dc62-2729-4139-b5e5-68709b534914 None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:34.798339 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cbe9a5-1299-4f20-b831-0e21394b9786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:34.798928 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cbe9a5-1299-4f20-b831-0e21394b9786 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:34.799201 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cbe9a5-1299-4f20-b831-0e21394b9786 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:34.799478 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cbe9a5-1299-4f20-b831-0e21394b9786 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:34.800189 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cbe9a5-1299-4f20-b831-0e21394b9786 None admin] RBAC: Adding query filter params (role.id=b870cd1acaa744e1ae1d58b9ac283821, scope.project.id=420b18725f2342159646fb86438318fd, user.id=cf0c99771661413aaa0f237af6b394ab) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:38:34.800488 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cbe9a5-1299-4f20-b831-0e21394b9786 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:38:34.801917 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cbe9a5-1299-4f20-b831-0e21394b9786 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:38:34.807519 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 641/1287] 10.222.0.55 () {56 vars in 1374 bytes} [Wed Aug 24 08:38:34 2022] GET /identity/v3/role_assignments?user.id=cf0c99771661413aaa0f237af6b394ab&scope.project.id=420b18725f2342159646fb86438318fd&role.id=b870cd1acaa744e1ae1d58b9ac283821 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:38:36.736339 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b580693-3711-46b6-a0f7-57d118f78c15 None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:36.736901 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b580693-3711-46b6-a0f7-57d118f78c15 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:36.736901 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b580693-3711-46b6-a0f7-57d118f78c15 None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:36.738262 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 647/1288] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:38:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:38:36.743898 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95fe60a-94b8-4361-b8da-e2e14a9c5617 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:36.745667 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95fe60a-94b8-4361-b8da-e2e14a9c5617 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:36.745667 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95fe60a-94b8-4361-b8da-e2e14a9c5617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:36.764199 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-d95fe60a-94b8-4361-b8da-e2e14a9c5617 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:38:36.766369 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-d95fe60a-94b8-4361-b8da-e2e14a9c5617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:38:36.795535 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:38:36.831938 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 642/1289] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:38:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:38:37.103745 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b580693-3711-46b6-a0f7-57d118f78c15 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:37.205278 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b580693-3711-46b6-a0f7-57d118f78c15 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:37.216408 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:38:37.240366 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33251b1-6716-40d0-9ab0-1b095ff7a939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:37.241013 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33251b1-6716-40d0-9ab0-1b095ff7a939 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:37.241288 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33251b1-6716-40d0-9ab0-1b095ff7a939 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:37.241558 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33251b1-6716-40d0-9ab0-1b095ff7a939 None admin] PATH_INFO: `/v3/services/placement` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:37.242131 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33251b1-6716-40d0-9ab0-1b095ff7a939 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:38:37.243475 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33251b1-6716-40d0-9ab0-1b095ff7a939 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:38:37.252647 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-d33251b1-6716-40d0-9ab0-1b095ff7a939 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 08:38:37.252647 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:38:37.252647 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 08:38:37.252647 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 08:38:37.252647 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 08:38:37.252647 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 08:38:37.252647 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 08:38:37.252647 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:38:37.252647 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 08:38:37.252647 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:38:37.252647 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:38:37.252647 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:38:37.252647 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:38:37.252647 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:38:37.252647 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:38:37.252647 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:38:37.252647 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:38:37.258592 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:38:37.258592 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:38:37.258592 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:38:37.258592 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:38:37.258592 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:38:37.258592 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:38:37.258592 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 08:38:37.258592 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 08:38:37.258592 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 08:38:37.258592 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 08:38:37.258592 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:38:37.258592 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:38:37.258592 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:38:37.258592 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:38:37.258592 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:38:37.258592 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:38:37.259522 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:38:37.259522 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:38:37.259522 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:38:37.259522 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:38:37.259522 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:38:37.259522 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:38:37.259522 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:38:37.259522 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:38:37.259522 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:38:37.259522 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:38:37.259522 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 08:38:37.259522 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:38:37.259522 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 08:38:37.259522 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:38:37.259522 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 648/1290] 10.222.0.55 () {56 vars in 1117 bytes} [Wed Aug 24 08:38:37 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:38:37.260478 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95fe60a-94b8-4361-b8da-e2e14a9c5617 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:37.364363 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95fe60a-94b8-4361-b8da-e2e14a9c5617 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:37.398245 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4d2d19-96b4-4bf9-8a95-d197b06487a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:37.398846 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4d2d19-96b4-4bf9-8a95-d197b06487a8 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:37.399123 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4d2d19-96b4-4bf9-8a95-d197b06487a8 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:37.399395 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4d2d19-96b4-4bf9-8a95-d197b06487a8 None admin] PATH_INFO: `/v3/services` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:37.400041 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4d2d19-96b4-4bf9-8a95-d197b06487a8 None admin] RBAC: Adding query filter params (name=placement) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:38:37.400316 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4d2d19-96b4-4bf9-8a95-d197b06487a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:38:37.401709 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4d2d19-96b4-4bf9-8a95-d197b06487a8 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:38:37.406857 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 643/1291] 10.222.0.55 () {56 vars in 1116 bytes} [Wed Aug 24 08:38:37 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:38:37.412760 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33251b1-6716-40d0-9ab0-1b095ff7a939 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:37.513159 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33251b1-6716-40d0-9ab0-1b095ff7a939 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:37.552022 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386d2a5f-9f32-4fce-8a6c-79a21c8764d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:37.552646 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386d2a5f-9f32-4fce-8a6c-79a21c8764d1 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:37.552927 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386d2a5f-9f32-4fce-8a6c-79a21c8764d1 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:37.553200 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386d2a5f-9f32-4fce-8a6c-79a21c8764d1 None admin] PATH_INFO: `/v3/services` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:37.553830 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386d2a5f-9f32-4fce-8a6c-79a21c8764d1 None admin] RBAC: Adding query filter params (type=placement) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:38:37.554132 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386d2a5f-9f32-4fce-8a6c-79a21c8764d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:38:37.555562 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386d2a5f-9f32-4fce-8a6c-79a21c8764d1 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:38:37.559878 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 649/1292] 10.222.0.55 () {56 vars in 1116 bytes} [Wed Aug 24 08:38:37 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:38:39.439279 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bdeecd-3827-4798-b277-27ba6e0eeda3 None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:39.439797 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bdeecd-3827-4798-b277-27ba6e0eeda3 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:39.439938 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bdeecd-3827-4798-b277-27ba6e0eeda3 None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:39.441265 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 644/1293] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:38:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:38:39.446652 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0556cd-c58c-461c-8c8f-389c938af33b None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:39.447040 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0556cd-c58c-461c-8c8f-389c938af33b None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:39.447139 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0556cd-c58c-461c-8c8f-389c938af33b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:39.465576 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-ee0556cd-c58c-461c-8c8f-389c938af33b None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:38:39.466759 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-ee0556cd-c58c-461c-8c8f-389c938af33b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:38:39.496358 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:38:39.535179 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 650/1294] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:38:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:38:39.800971 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bdeecd-3827-4798-b277-27ba6e0eeda3 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:39.832224 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:38:39.900416 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bdeecd-3827-4798-b277-27ba6e0eeda3 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:39.934329 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3a4e95-a661-4d12-a776-ef802fc58cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:39.934930 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3a4e95-a661-4d12-a776-ef802fc58cde None admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:39.935205 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3a4e95-a661-4d12-a776-ef802fc58cde None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:39.935470 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3a4e95-a661-4d12-a776-ef802fc58cde None admin] PATH_INFO: `/v3/services` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:39.936102 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3a4e95-a661-4d12-a776-ef802fc58cde None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:38:39.938008 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3a4e95-a661-4d12-a776-ef802fc58cde None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:38:40.088547 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 645/1295] 10.222.0.55 () {60 vars in 1141 bytes} [Wed Aug 24 08:38:39 2022] POST /identity/v3/services => generated 246 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:38:42.011688 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d680f0f-f6cb-4ade-a7f9-c243169d4fb4 None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:42.012279 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d680f0f-f6cb-4ade-a7f9-c243169d4fb4 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:42.012279 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d680f0f-f6cb-4ade-a7f9-c243169d4fb4 None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:42.013623 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 651/1296] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:38:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:38:42.019057 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908630a9-0f6d-4c40-9638-5e27760f9211 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:42.019475 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908630a9-0f6d-4c40-9638-5e27760f9211 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:42.019575 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908630a9-0f6d-4c40-9638-5e27760f9211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:42.037747 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-908630a9-0f6d-4c40-9638-5e27760f9211 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:38:42.038919 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-908630a9-0f6d-4c40-9638-5e27760f9211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:38:42.112479 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 646/1297] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:38:42 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:38:42.379709 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d680f0f-f6cb-4ade-a7f9-c243169d4fb4 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:42.485062 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d680f0f-f6cb-4ade-a7f9-c243169d4fb4 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:42.518421 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e14ab3-7a67-4efd-a59b-a59aa08731f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:42.519044 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e14ab3-7a67-4efd-a59b-a59aa08731f2 None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:42.519324 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e14ab3-7a67-4efd-a59b-a59aa08731f2 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:42.519596 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e14ab3-7a67-4efd-a59b-a59aa08731f2 None admin] PATH_INFO: `/v3/services/placement` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:42.520171 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e14ab3-7a67-4efd-a59b-a59aa08731f2 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:38:42.521546 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e14ab3-7a67-4efd-a59b-a59aa08731f2 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:38:42.530038 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-63e14ab3-7a67-4efd-a59b-a59aa08731f2 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 08:38:42.530038 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:38:42.530038 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 08:38:42.530038 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 08:38:42.530038 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 08:38:42.530038 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 08:38:42.530038 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 08:38:42.530038 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:38:42.530038 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 08:38:42.530038 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:38:42.530038 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:38:42.530038 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:38:42.530038 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:38:42.530038 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:38:42.530038 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:38:42.530038 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:38:42.530038 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:38:42.531947 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:38:42.531947 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:38:42.531947 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:38:42.531947 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:38:42.531947 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:38:42.531947 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:38:42.531947 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 08:38:42.531947 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 08:38:42.531947 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 08:38:42.531947 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 08:38:42.531947 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:38:42.531947 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:38:42.531947 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:38:42.531947 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:38:42.531947 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:38:42.531947 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:38:42.533152 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:38:42.533152 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:38:42.533152 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:38:42.533152 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:38:42.533152 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:38:42.533152 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:38:42.533152 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:38:42.533152 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:38:42.533152 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:38:42.533152 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:38:42.533152 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 08:38:42.533152 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:38:42.533152 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 08:38:42.533152 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:38:42.533152 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 652/1298] 10.222.0.55 () {56 vars in 1117 bytes} [Wed Aug 24 08:38:42 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:38:42.537764 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908630a9-0f6d-4c40-9638-5e27760f9211 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:42.555577 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:38:42.632034 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908630a9-0f6d-4c40-9638-5e27760f9211 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:42.664794 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e08636-6b3b-499e-96fe-04cb5db3294c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:42.665384 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e08636-6b3b-499e-96fe-04cb5db3294c None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:42.665657 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e08636-6b3b-499e-96fe-04cb5db3294c None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:42.665924 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e08636-6b3b-499e-96fe-04cb5db3294c None admin] PATH_INFO: `/v3/services` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:42.666538 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e08636-6b3b-499e-96fe-04cb5db3294c None admin] RBAC: Adding query filter params (name=placement) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:38:42.666831 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e08636-6b3b-499e-96fe-04cb5db3294c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:38:42.668176 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e08636-6b3b-499e-96fe-04cb5db3294c None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:38:42.673194 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 647/1299] 10.222.0.55 () {56 vars in 1116 bytes} [Wed Aug 24 08:38:42 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:38:42.679286 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e14ab3-7a67-4efd-a59b-a59aa08731f2 None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:42.712188 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:38:42.778398 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e14ab3-7a67-4efd-a59b-a59aa08731f2 None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:42.810118 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7b4509-7e32-4a56-9ccc-1c4e55d2ce1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:42.815685 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7b4509-7e32-4a56-9ccc-1c4e55d2ce1d None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:42.815685 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7b4509-7e32-4a56-9ccc-1c4e55d2ce1d None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:42.815685 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7b4509-7e32-4a56-9ccc-1c4e55d2ce1d None admin] PATH_INFO: `/v3/endpoints` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:42.815685 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7b4509-7e32-4a56-9ccc-1c4e55d2ce1d None admin] RBAC: Adding query filter params (interface=public, service_id=a84de0e13043404ea1e9405f6b0e7dd8, region_id=RegionOne) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:38:42.815685 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7b4509-7e32-4a56-9ccc-1c4e55d2ce1d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:38:42.815685 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7b4509-7e32-4a56-9ccc-1c4e55d2ce1d None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:38:42.819220 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 653/1300] 10.222.0.55 () {56 vars in 1251 bytes} [Wed Aug 24 08:38:42 2022] GET /identity/v3/endpoints?service_id=a84de0e13043404ea1e9405f6b0e7dd8&interface=public®ion_id=RegionOne => generated 194 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:38:42.825169 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e08636-6b3b-499e-96fe-04cb5db3294c None admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:42.924201 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e08636-6b3b-499e-96fe-04cb5db3294c None admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:42.957354 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb1dd0d-76b5-46c5-91c3-5c1f6e41019e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:42.957953 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb1dd0d-76b5-46c5-91c3-5c1f6e41019e None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:42.958228 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb1dd0d-76b5-46c5-91c3-5c1f6e41019e None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:42.958499 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb1dd0d-76b5-46c5-91c3-5c1f6e41019e None admin] PATH_INFO: `/v3/services` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:42.959094 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb1dd0d-76b5-46c5-91c3-5c1f6e41019e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:38:42.960444 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb1dd0d-76b5-46c5-91c3-5c1f6e41019e None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:38:42.967045 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 648/1301] 10.222.0.55 () {56 vars in 1087 bytes} [Wed Aug 24 08:38:42 2022] GET /identity/v3/services => generated 2191 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:38:43.075158 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:38:44.862422 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e01b4a5-9f6e-45db-bf3a-f438522dac56 None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:44.862996 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e01b4a5-9f6e-45db-bf3a-f438522dac56 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:44.862996 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e01b4a5-9f6e-45db-bf3a-f438522dac56 None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:44.864261 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 654/1302] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:38:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:38:44.869640 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c2742-7caf-43e4-97dc-8c5d2314ef76 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:44.870084 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c2742-7caf-43e4-97dc-8c5d2314ef76 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:44.874952 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c2742-7caf-43e4-97dc-8c5d2314ef76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:44.888726 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-960c2742-7caf-43e4-97dc-8c5d2314ef76 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:38:44.889916 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-960c2742-7caf-43e4-97dc-8c5d2314ef76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:38:44.958270 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 649/1303] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:38:44 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:38:45.223992 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e01b4a5-9f6e-45db-bf3a-f438522dac56 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:45.322584 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:38:45.329688 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e01b4a5-9f6e-45db-bf3a-f438522dac56 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:45.363638 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18843b76-3bfc-4ae3-9235-ad3b3812a53e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:45.364258 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18843b76-3bfc-4ae3-9235-ad3b3812a53e None admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:45.364532 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18843b76-3bfc-4ae3-9235-ad3b3812a53e None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:45.364870 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18843b76-3bfc-4ae3-9235-ad3b3812a53e None admin] PATH_INFO: `/v3/services/placement` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:45.365452 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18843b76-3bfc-4ae3-9235-ad3b3812a53e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:38:45.366798 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18843b76-3bfc-4ae3-9235-ad3b3812a53e None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:38:45.375304 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application [None req-18843b76-3bfc-4ae3-9235-ad3b3812a53e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 08:38:45.375304 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:38:45.375304 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 08:38:45.375304 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 08:38:45.375304 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 08:38:45.375304 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 08:38:45.375304 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 08:38:45.375304 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:38:45.375304 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 08:38:45.375304 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:38:45.375304 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:38:45.375304 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:38:45.375304 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:38:45.375304 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:38:45.375304 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:38:45.375304 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:38:45.375304 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:38:45.380730 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:38:45.380730 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:38:45.380730 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:38:45.380730 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:38:45.380730 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:38:45.380730 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:38:45.380730 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 08:38:45.380730 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 08:38:45.380730 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 08:38:45.380730 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 08:38:45.380730 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:38:45.380730 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:38:45.380730 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:38:45.380730 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:38:45.380730 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:38:45.380730 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:38:45.381866 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:38:45.381866 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application with Lock( Aug 24 08:38:45.381866 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:38:45.381866 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:38:45.381866 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:38:45.381866 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:38:45.381866 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:38:45.381866 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:38:45.381866 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:38:45.381866 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:38:45.381866 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 08:38:45.381866 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:38:45.381866 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 08:38:45.381866 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: ERROR keystone.server.flask.application Aug 24 08:38:45.381866 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 655/1304] 10.222.0.55 () {56 vars in 1117 bytes} [Wed Aug 24 08:38:45 2022] GET /identity/v3/services/placement => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:38:45.382877 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960c2742-7caf-43e4-97dc-8c5d2314ef76 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:45.466884 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:38:45.489943 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960c2742-7caf-43e4-97dc-8c5d2314ef76 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:45.523212 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b04862-2a5f-45b3-9c6a-6ca076d8edb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:45.523815 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b04862-2a5f-45b3-9c6a-6ca076d8edb3 None admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:45.524095 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b04862-2a5f-45b3-9c6a-6ca076d8edb3 None admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:45.524368 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b04862-2a5f-45b3-9c6a-6ca076d8edb3 None admin] PATH_INFO: `/v3/services` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:45.525057 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b04862-2a5f-45b3-9c6a-6ca076d8edb3 None admin] RBAC: Adding query filter params (name=placement) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:38:45.525360 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b04862-2a5f-45b3-9c6a-6ca076d8edb3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:38:45.526762 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b04862-2a5f-45b3-9c6a-6ca076d8edb3 None admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:38:45.532325 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 650/1305] 10.222.0.55 () {56 vars in 1116 bytes} [Wed Aug 24 08:38:45 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:38:45.538141 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18843b76-3bfc-4ae3-9235-ad3b3812a53e None admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:38:45.638877 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18843b76-3bfc-4ae3-9235-ad3b3812a53e None admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:38:45.672757 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f443bb0d-184d-45ce-a035-fab8fbcdf368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:38:45.673356 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f443bb0d-184d-45ce-a035-fab8fbcdf368 None admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:38:45.673706 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f443bb0d-184d-45ce-a035-fab8fbcdf368 None admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:38:45.673908 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f443bb0d-184d-45ce-a035-fab8fbcdf368 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:38:45.674559 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f443bb0d-184d-45ce-a035-fab8fbcdf368 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:38:45.675935 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f443bb0d-184d-45ce-a035-fab8fbcdf368 None admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:38:45.837593 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 656/1306] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 08:38:45 2022] POST /identity/v3/endpoints => generated 331 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:38:55.675215 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=30 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:38:55.832905 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:47:06.445375 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf16cb0-1bc3-432c-bcc7-9406f72c4361 None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:06.446020 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf16cb0-1bc3-432c-bcc7-9406f72c4361 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:06.446020 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf16cb0-1bc3-432c-bcc7-9406f72c4361 None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:06.447662 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 651/1307] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:47:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:47:06.456231 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a554d0ac-b7ec-4e28-bf1c-b8067a28d63d None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:06.456231 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a554d0ac-b7ec-4e28-bf1c-b8067a28d63d None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:06.456231 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a554d0ac-b7ec-4e28-bf1c-b8067a28d63d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:06.610048 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-a554d0ac-b7ec-4e28-bf1c-b8067a28d63d None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:47:06.612606 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-a554d0ac-b7ec-4e28-bf1c-b8067a28d63d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:06.823654 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 657/1308] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:47:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:47:07.128369 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec4352f-e97a-4ee7-bbbc-0d772bdaaaf3 None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:07.128369 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec4352f-e97a-4ee7-bbbc-0d772bdaaaf3 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:07.129168 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec4352f-e97a-4ee7-bbbc-0d772bdaaaf3 None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:07.132028 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 652/1309] 10.222.0.55 () {54 vars in 965 bytes} [Wed Aug 24 08:47:07 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:47:07.140137 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1190c8b-1e92-4fc2-8e3d-e21eb077e508 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:07.140433 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1190c8b-1e92-4fc2-8e3d-e21eb077e508 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:07.140754 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1190c8b-1e92-4fc2-8e3d-e21eb077e508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:07.231494 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-a1190c8b-1e92-4fc2-8e3d-e21eb077e508 None None] MFA Rules not processed for user `28b58388d1664510b0b507323bca5ae8`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:47:07.232725 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-a1190c8b-1e92-4fc2-8e3d-e21eb077e508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:07.255523 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:47:07.318138 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 658/1310] 10.222.0.55 () {58 vars in 1062 bytes} [Wed Aug 24 08:47:07 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:47:07.328457 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec4352f-e97a-4ee7-bbbc-0d772bdaaaf3 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:47:07.330233 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-5ec4352f-e97a-4ee7-bbbc-0d772bdaaaf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:07.480238 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec4352f-e97a-4ee7-bbbc-0d772bdaaaf3 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:47:07.594401 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284c296f-651b-4c50-872f-617f01dd77d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:47:07.595287 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284c296f-651b-4c50-872f-617f01dd77d0 service neutron] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:07.595552 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284c296f-651b-4c50-872f-617f01dd77d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:07.595914 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284c296f-651b-4c50-872f-617f01dd77d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:07.597781 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-284c296f-651b-4c50-872f-617f01dd77d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:07.669833 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:47:07.745533 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284c296f-651b-4c50-872f-617f01dd77d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:47:07.747799 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284c296f-651b-4c50-872f-617f01dd77d0 service neutron] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:47:07.773625 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 653/1311] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:47:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:47:10.185424 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a424cba1-6b78-499c-8748-e7f11602e4d3 None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:10.186138 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a424cba1-6b78-499c-8748-e7f11602e4d3 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:10.186138 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a424cba1-6b78-499c-8748-e7f11602e4d3 None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:10.187177 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 659/1312] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:47:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:47:10.193253 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad0cabd-0102-4759-80b4-c66d4129dc3b None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:10.193771 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad0cabd-0102-4759-80b4-c66d4129dc3b None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:10.193935 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad0cabd-0102-4759-80b4-c66d4129dc3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:10.286982 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-cad0cabd-0102-4759-80b4-c66d4129dc3b None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:47:10.288214 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-cad0cabd-0102-4759-80b4-c66d4129dc3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:10.410033 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 654/1313] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:47:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:47:10.721401 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56772b14-dba1-40fb-8331-02bcc6e8f0c8 None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:10.721725 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56772b14-dba1-40fb-8331-02bcc6e8f0c8 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:10.721931 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56772b14-dba1-40fb-8331-02bcc6e8f0c8 None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:10.723331 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 660/1314] 10.222.0.55 () {54 vars in 965 bytes} [Wed Aug 24 08:47:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:47:10.731278 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02e10e8-587f-4574-bd26-d62f8009c0d0 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:10.731731 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02e10e8-587f-4574-bd26-d62f8009c0d0 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:10.732189 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02e10e8-587f-4574-bd26-d62f8009c0d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:10.751606 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:47:10.836586 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-c02e10e8-587f-4574-bd26-d62f8009c0d0 None None] MFA Rules not processed for user `28b58388d1664510b0b507323bca5ae8`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:47:10.837834 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-c02e10e8-587f-4574-bd26-d62f8009c0d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:10.915967 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 655/1315] 10.222.0.55 () {58 vars in 1062 bytes} [Wed Aug 24 08:47:10 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:47:10.925017 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56772b14-dba1-40fb-8331-02bcc6e8f0c8 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:47:10.926717 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-56772b14-dba1-40fb-8331-02bcc6e8f0c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:11.037215 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56772b14-dba1-40fb-8331-02bcc6e8f0c8 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:47:11.109900 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:47:11.112229 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589a87e9-c8d0-418d-b076-3792e8778296 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:47:11.112865 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589a87e9-c8d0-418d-b076-3792e8778296 service neutron] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:11.113149 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589a87e9-c8d0-418d-b076-3792e8778296 service neutron] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:11.113409 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589a87e9-c8d0-418d-b076-3792e8778296 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:11.115013 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-589a87e9-c8d0-418d-b076-3792e8778296 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:11.227463 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589a87e9-c8d0-418d-b076-3792e8778296 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:47:11.229000 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589a87e9-c8d0-418d-b076-3792e8778296 service neutron] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:47:11.250560 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 661/1316] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:47:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:47:13.908824 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7145ea4-db48-4119-b1f0-582421425e16 None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:13.909466 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7145ea4-db48-4119-b1f0-582421425e16 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:13.909466 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7145ea4-db48-4119-b1f0-582421425e16 None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:13.910811 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 656/1317] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:47:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:47:13.916524 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23ad6ac-46c1-4aec-a136-6b42777f8e45 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:13.917000 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23ad6ac-46c1-4aec-a136-6b42777f8e45 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:13.917092 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23ad6ac-46c1-4aec-a136-6b42777f8e45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:13.986886 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-b23ad6ac-46c1-4aec-a136-6b42777f8e45 None None] MFA Rules not processed for user `07914530ee0141de9cc889c157da5a4c`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:47:13.988115 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-b23ad6ac-46c1-4aec-a136-6b42777f8e45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:14.041834 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:47:14.092127 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 662/1318] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:47:13 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:47:14.386801 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7145ea4-db48-4119-b1f0-582421425e16 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:47:14.388484 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-e7145ea4-db48-4119-b1f0-582421425e16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:14.495409 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7145ea4-db48-4119-b1f0-582421425e16 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:47:14.498735 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:47:14.575044 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dc96e5-7414-4df6-b2c4-885978717dc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:47:14.576235 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dc96e5-7414-4df6-b2c4-885978717dc9 service neutron] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:14.576557 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dc96e5-7414-4df6-b2c4-885978717dc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:14.576863 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dc96e5-7414-4df6-b2c4-885978717dc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:14.578676 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-c9dc96e5-7414-4df6-b2c4-885978717dc9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:14.691357 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dc96e5-7414-4df6-b2c4-885978717dc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:47:14.692825 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dc96e5-7414-4df6-b2c4-885978717dc9 service neutron] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:47:14.712305 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 657/1319] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:47:14 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:47:17.370625 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94be6c60-dca1-4b42-a3d3-562e6bea652f None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:17.371194 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94be6c60-dca1-4b42-a3d3-562e6bea652f None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:17.371194 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94be6c60-dca1-4b42-a3d3-562e6bea652f None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:17.373139 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 663/1320] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:47:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:47:17.378558 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712a1563-4d44-48f3-a755-5e4791091bd9 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:17.378961 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712a1563-4d44-48f3-a755-5e4791091bd9 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:17.379107 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712a1563-4d44-48f3-a755-5e4791091bd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:17.419729 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:47:17.447309 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-712a1563-4d44-48f3-a755-5e4791091bd9 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:47:17.448889 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-712a1563-4d44-48f3-a755-5e4791091bd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:17.545522 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 658/1321] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:47:17 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:47:17.829186 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94be6c60-dca1-4b42-a3d3-562e6bea652f None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:47:17.830713 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-94be6c60-dca1-4b42-a3d3-562e6bea652f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:17.935741 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94be6c60-dca1-4b42-a3d3-562e6bea652f None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:47:18.008555 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc47701-6e3b-4d75-bed6-ab289e900501 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:47:18.009166 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc47701-6e3b-4d75-bed6-ab289e900501 service neutron] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:18.009445 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc47701-6e3b-4d75-bed6-ab289e900501 service neutron] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:18.009714 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc47701-6e3b-4d75-bed6-ab289e900501 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:18.011298 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-3cc47701-6e3b-4d75-bed6-ab289e900501 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:18.019556 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:47:18.122243 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc47701-6e3b-4d75-bed6-ab289e900501 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:47:18.123601 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc47701-6e3b-4d75-bed6-ab289e900501 service neutron] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:47:18.143972 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 664/1322] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:47:17 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:47:21.104797 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6873ed6-5a49-4921-ba1e-c23eb12a7a8f None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:21.105348 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6873ed6-5a49-4921-ba1e-c23eb12a7a8f None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:21.105348 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6873ed6-5a49-4921-ba1e-c23eb12a7a8f None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:21.107502 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 659/1323] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:47:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:47:21.113540 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23de4ff2-f57b-4749-99e7-5fe714f07169 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:21.114040 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23de4ff2-f57b-4749-99e7-5fe714f07169 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:21.114214 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23de4ff2-f57b-4749-99e7-5fe714f07169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:21.186106 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-23de4ff2-f57b-4749-99e7-5fe714f07169 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:47:21.187334 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-23de4ff2-f57b-4749-99e7-5fe714f07169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:21.249213 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:47:21.292293 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 665/1324] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:47:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:47:21.577764 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6873ed6-5a49-4921-ba1e-c23eb12a7a8f None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:47:21.579517 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-a6873ed6-5a49-4921-ba1e-c23eb12a7a8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:21.694111 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6873ed6-5a49-4921-ba1e-c23eb12a7a8f None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:47:21.707026 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:47:21.783054 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869ceb53-cfec-47df-8e4d-9436dff4a1b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:47:21.783689 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869ceb53-cfec-47df-8e4d-9436dff4a1b5 service neutron] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:21.783965 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869ceb53-cfec-47df-8e4d-9436dff4a1b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:21.784233 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869ceb53-cfec-47df-8e4d-9436dff4a1b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:21.786169 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-869ceb53-cfec-47df-8e4d-9436dff4a1b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:21.916962 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869ceb53-cfec-47df-8e4d-9436dff4a1b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:47:21.918282 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869ceb53-cfec-47df-8e4d-9436dff4a1b5 service neutron] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:47:21.939112 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 660/1325] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:47:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:47:23.948641 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cea3efd-888d-4ed1-9cde-0097c9971966 None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:23.949201 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cea3efd-888d-4ed1-9cde-0097c9971966 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:23.949201 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cea3efd-888d-4ed1-9cde-0097c9971966 None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:23.951031 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 666/1326] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:47:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:47:23.956622 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae51e6f-a2e2-4ff6-a1e3-711b68ab27a6 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:23.957095 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae51e6f-a2e2-4ff6-a1e3-711b68ab27a6 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:23.957210 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae51e6f-a2e2-4ff6-a1e3-711b68ab27a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:24.007772 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:47:24.027402 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-aae51e6f-a2e2-4ff6-a1e3-711b68ab27a6 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:47:24.029190 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-aae51e6f-a2e2-4ff6-a1e3-711b68ab27a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:24.129500 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 661/1327] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:47:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:47:24.409177 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cea3efd-888d-4ed1-9cde-0097c9971966 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:47:24.410716 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-6cea3efd-888d-4ed1-9cde-0097c9971966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:24.523900 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cea3efd-888d-4ed1-9cde-0097c9971966 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:47:24.603319 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70c3f78-947d-45c5-a01d-92ee450e46d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:47:24.603941 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70c3f78-947d-45c5-a01d-92ee450e46d1 service neutron] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:24.604218 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70c3f78-947d-45c5-a01d-92ee450e46d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:24.604528 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70c3f78-947d-45c5-a01d-92ee450e46d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:24.606203 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-f70c3f78-947d-45c5-a01d-92ee450e46d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:24.624160 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:47:24.729268 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70c3f78-947d-45c5-a01d-92ee450e46d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:47:24.730713 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70c3f78-947d-45c5-a01d-92ee450e46d1 service neutron] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:47:24.749635 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 667/1328] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:47:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:47:26.718211 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65f3003-cb13-45c6-9c8b-76c24105c9d7 None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:26.718773 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65f3003-cb13-45c6-9c8b-76c24105c9d7 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:26.718773 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65f3003-cb13-45c6-9c8b-76c24105c9d7 None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:26.720104 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 662/1329] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:47:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:47:26.725736 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebcda86-6b64-4450-a260-6201cf78e3c7 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:26.726188 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebcda86-6b64-4450-a260-6201cf78e3c7 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:26.726289 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebcda86-6b64-4450-a260-6201cf78e3c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:26.799375 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-5ebcda86-6b64-4450-a260-6201cf78e3c7 None None] MFA Rules not processed for user `07914530ee0141de9cc889c157da5a4c`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:47:26.800690 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-5ebcda86-6b64-4450-a260-6201cf78e3c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:26.876583 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:47:26.913393 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 668/1330] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:47:26 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:47:27.210661 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65f3003-cb13-45c6-9c8b-76c24105c9d7 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:47:27.212196 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-d65f3003-cb13-45c6-9c8b-76c24105c9d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:27.340968 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65f3003-cb13-45c6-9c8b-76c24105c9d7 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:47:27.362108 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:47:27.422615 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36be237d-1a11-44fd-9a48-7f3f4cfe0598 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:47:27.423227 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36be237d-1a11-44fd-9a48-7f3f4cfe0598 service neutron] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:27.423505 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36be237d-1a11-44fd-9a48-7f3f4cfe0598 service neutron] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:27.423778 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36be237d-1a11-44fd-9a48-7f3f4cfe0598 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:27.425452 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-36be237d-1a11-44fd-9a48-7f3f4cfe0598 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:27.537136 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36be237d-1a11-44fd-9a48-7f3f4cfe0598 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:47:27.538505 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36be237d-1a11-44fd-9a48-7f3f4cfe0598 service neutron] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:47:27.557612 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 663/1331] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:47:27 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:47:29.963361 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d87f1-ed2f-4bce-b119-fbeacf45b923 None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:29.964019 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d87f1-ed2f-4bce-b119-fbeacf45b923 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:29.964019 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d87f1-ed2f-4bce-b119-fbeacf45b923 None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:29.965396 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 669/1332] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:47:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:47:29.971220 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713811be-85fc-4c98-82ce-e07838be1660 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:29.971980 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713811be-85fc-4c98-82ce-e07838be1660 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:29.972152 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713811be-85fc-4c98-82ce-e07838be1660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:30.048876 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:47:30.053882 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-713811be-85fc-4c98-82ce-e07838be1660 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:47:30.055113 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-713811be-85fc-4c98-82ce-e07838be1660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:30.164445 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 664/1333] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:47:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:47:30.445637 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579d87f1-ed2f-4bce-b119-fbeacf45b923 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:47:30.447184 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-579d87f1-ed2f-4bce-b119-fbeacf45b923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:30.558076 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579d87f1-ed2f-4bce-b119-fbeacf45b923 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:47:30.634024 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd538ed-b012-410f-8716-4c0252274045 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:47:30.634635 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd538ed-b012-410f-8716-4c0252274045 service neutron] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:30.634918 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd538ed-b012-410f-8716-4c0252274045 service neutron] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:30.635187 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd538ed-b012-410f-8716-4c0252274045 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:30.636824 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-ffd538ed-b012-410f-8716-4c0252274045 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:30.666048 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:47:30.752986 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd538ed-b012-410f-8716-4c0252274045 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:47:30.754477 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd538ed-b012-410f-8716-4c0252274045 service neutron] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:47:30.774359 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 670/1334] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:47:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:47:32.709364 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb3470c-3d73-4f54-9dc5-91936c0f0143 None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:32.709917 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb3470c-3d73-4f54-9dc5-91936c0f0143 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:32.709917 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb3470c-3d73-4f54-9dc5-91936c0f0143 None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:32.711369 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 665/1335] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:47:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:47:32.717224 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782e4812-9ff7-4764-bca3-ca403f078eba None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:32.717651 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782e4812-9ff7-4764-bca3-ca403f078eba None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:32.717772 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782e4812-9ff7-4764-bca3-ca403f078eba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:32.786041 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-782e4812-9ff7-4764-bca3-ca403f078eba None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:47:32.787283 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-782e4812-9ff7-4764-bca3-ca403f078eba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:32.865194 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:47:32.888064 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 671/1336] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:47:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:47:33.170927 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb3470c-3d73-4f54-9dc5-91936c0f0143 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:47:33.172645 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-ffb3470c-3d73-4f54-9dc5-91936c0f0143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:33.283020 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb3470c-3d73-4f54-9dc5-91936c0f0143 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:47:33.311890 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:47:33.359475 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afbf967-76a4-443d-bb31-68d73ce14a2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:47:33.360007 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afbf967-76a4-443d-bb31-68d73ce14a2d service neutron] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:33.360303 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afbf967-76a4-443d-bb31-68d73ce14a2d service neutron] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:33.360602 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afbf967-76a4-443d-bb31-68d73ce14a2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:33.362400 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-2afbf967-76a4-443d-bb31-68d73ce14a2d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:33.474641 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afbf967-76a4-443d-bb31-68d73ce14a2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:47:33.476059 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afbf967-76a4-443d-bb31-68d73ce14a2d service neutron] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:47:33.494753 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 666/1337] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:47:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:47:36.530416 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7474bac0-5002-423e-876a-8cabbf0285f8 None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:36.530933 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7474bac0-5002-423e-876a-8cabbf0285f8 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:36.530933 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7474bac0-5002-423e-876a-8cabbf0285f8 None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:36.532287 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 672/1338] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:47:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:47:36.538779 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7527e098-2f24-43b6-9915-e8a435c3131d None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:36.539308 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7527e098-2f24-43b6-9915-e8a435c3131d None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:36.539422 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7527e098-2f24-43b6-9915-e8a435c3131d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:36.622682 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-7527e098-2f24-43b6-9915-e8a435c3131d None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:47:36.623995 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-7527e098-2f24-43b6-9915-e8a435c3131d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:36.633104 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:47:36.728128 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 667/1339] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:47:36 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:47:37.010107 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7474bac0-5002-423e-876a-8cabbf0285f8 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:47:37.011649 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-7474bac0-5002-423e-876a-8cabbf0285f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:37.115414 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7474bac0-5002-423e-876a-8cabbf0285f8 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:47:37.189846 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e082b82-a3ed-48e9-9a21-130b7610d5f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:47:37.190447 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e082b82-a3ed-48e9-9a21-130b7610d5f8 service neutron] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:37.190721 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e082b82-a3ed-48e9-9a21-130b7610d5f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:37.190988 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e082b82-a3ed-48e9-9a21-130b7610d5f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:37.192614 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-2e082b82-a3ed-48e9-9a21-130b7610d5f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:37.232806 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:47:37.307892 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e082b82-a3ed-48e9-9a21-130b7610d5f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:47:37.309907 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e082b82-a3ed-48e9-9a21-130b7610d5f8 service neutron] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:47:37.331261 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 673/1340] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:47:37 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:47:41.984051 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d550b5-63d5-48b7-b23a-0c98d749beef None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:41.984955 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d550b5-63d5-48b7-b23a-0c98d749beef None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:41.984955 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d550b5-63d5-48b7-b23a-0c98d749beef None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:41.985995 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 668/1341] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:47:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:47:41.993174 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4ce828-0be7-4891-a3fc-f026418ea007 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:41.993927 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4ce828-0be7-4891-a3fc-f026418ea007 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:41.994359 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4ce828-0be7-4891-a3fc-f026418ea007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:42.119810 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-8e4ce828-0be7-4891-a3fc-f026418ea007 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:47:42.121773 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-8e4ce828-0be7-4891-a3fc-f026418ea007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:42.275113 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:47:42.296748 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 674/1342] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:47:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:47:42.787208 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d550b5-63d5-48b7-b23a-0c98d749beef None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:47:42.788776 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-63d550b5-63d5-48b7-b23a-0c98d749beef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:42.941508 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d550b5-63d5-48b7-b23a-0c98d749beef None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:47:42.997007 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:47:43.050982 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfebd897-ddec-4d69-95ba-0e51717f4260 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:47:43.051566 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfebd897-ddec-4d69-95ba-0e51717f4260 service neutron] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:43.051888 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfebd897-ddec-4d69-95ba-0e51717f4260 service neutron] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:43.052205 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfebd897-ddec-4d69-95ba-0e51717f4260 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:43.054115 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-dfebd897-ddec-4d69-95ba-0e51717f4260 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:43.201731 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfebd897-ddec-4d69-95ba-0e51717f4260 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:47:43.204181 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfebd897-ddec-4d69-95ba-0e51717f4260 service neutron] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:47:43.236383 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 669/1343] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:47:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:47:46.563798 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e031165c-cec4-4aad-b4c4-6e849ffa257f None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:46.564648 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e031165c-cec4-4aad-b4c4-6e849ffa257f None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:46.564648 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e031165c-cec4-4aad-b4c4-6e849ffa257f None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:46.565819 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 675/1344] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:47:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:47:46.572336 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff45136-45b6-4b8a-8a89-9836582c05f3 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:46.572934 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff45136-45b6-4b8a-8a89-9836582c05f3 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:46.573347 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff45136-45b6-4b8a-8a89-9836582c05f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:46.663385 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-fff45136-45b6-4b8a-8a89-9836582c05f3 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:47:46.664641 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-fff45136-45b6-4b8a-8a89-9836582c05f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:46.683244 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:47:46.786280 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 670/1345] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:47:46 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:47:47.073299 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e031165c-cec4-4aad-b4c4-6e849ffa257f None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:47:47.076321 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-e031165c-cec4-4aad-b4c4-6e849ffa257f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:47.222800 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e031165c-cec4-4aad-b4c4-6e849ffa257f None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:47:47.328576 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e1a649-eff9-4792-926a-a10152e641e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:47:47.329188 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e1a649-eff9-4792-926a-a10152e641e0 service neutron] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:47.329467 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e1a649-eff9-4792-926a-a10152e641e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:47.329733 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e1a649-eff9-4792-926a-a10152e641e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:47.331378 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-f4e1a649-eff9-4792-926a-a10152e641e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:47.383858 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:47:47.456128 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e1a649-eff9-4792-926a-a10152e641e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:47:47.457510 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e1a649-eff9-4792-926a-a10152e641e0 service neutron] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:47:47.479192 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 676/1346] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:47:47 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:47:51.790037 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19af9754-f7e7-472a-979c-30444f47e580 None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:51.790568 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19af9754-f7e7-472a-979c-30444f47e580 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:51.790700 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19af9754-f7e7-472a-979c-30444f47e580 None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:51.791986 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 671/1347] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:47:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:47:51.797675 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe686fc-9445-496c-9d7f-8f506fa643d9 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:51.798124 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe686fc-9445-496c-9d7f-8f506fa643d9 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:51.798216 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe686fc-9445-496c-9d7f-8f506fa643d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:51.869720 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-2fe686fc-9445-496c-9d7f-8f506fa643d9 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:47:51.871007 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-2fe686fc-9445-496c-9d7f-8f506fa643d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:51.979915 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:47:51.984222 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 677/1348] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:47:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:47:52.267042 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19af9754-f7e7-472a-979c-30444f47e580 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:47:52.268896 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-19af9754-f7e7-472a-979c-30444f47e580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:52.388465 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19af9754-f7e7-472a-979c-30444f47e580 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:47:52.439154 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:47:52.464950 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb36c2cc-d83b-42fa-a017-8d7c0c97ac3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:47:52.466870 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb36c2cc-d83b-42fa-a017-8d7c0c97ac3d service neutron] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:52.466870 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb36c2cc-d83b-42fa-a017-8d7c0c97ac3d service neutron] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:52.466870 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb36c2cc-d83b-42fa-a017-8d7c0c97ac3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:52.467715 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-bb36c2cc-d83b-42fa-a017-8d7c0c97ac3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:52.578459 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb36c2cc-d83b-42fa-a017-8d7c0c97ac3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:47:52.579807 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb36c2cc-d83b-42fa-a017-8d7c0c97ac3d service neutron] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:47:52.598367 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 672/1349] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:47:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:47:57.612776 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66b9d3d-689d-4a09-91bb-c05bb8c6e0c5 None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:57.614480 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66b9d3d-689d-4a09-91bb-c05bb8c6e0c5 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:57.614480 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66b9d3d-689d-4a09-91bb-c05bb8c6e0c5 None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:57.617094 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 678/1350] 10.222.0.55 () {54 vars in 921 bytes} [Wed Aug 24 08:47:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:47:57.625358 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ce652-fbfe-4948-b080-17e6f7a768ea None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:57.625982 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ce652-fbfe-4948-b080-17e6f7a768ea None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:57.625982 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ce652-fbfe-4948-b080-17e6f7a768ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:57.732014 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-387ce652-fbfe-4948-b080-17e6f7a768ea None None] MFA Rules not processed for user `cf0c99771661413aaa0f237af6b394ab`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:47:57.733500 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-387ce652-fbfe-4948-b080-17e6f7a768ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:57.754571 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:47:57.889022 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 673/1351] 10.222.0.55 () {58 vars in 1018 bytes} [Wed Aug 24 08:47:57 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:48:02.237595 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86555ad-ae3d-4e88-a21b-1f1d779939be None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:02.238134 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86555ad-ae3d-4e88-a21b-1f1d779939be None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:02.238219 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86555ad-ae3d-4e88-a21b-1f1d779939be None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:02.240422 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 679/1352] 10.222.0.55 () {54 vars in 921 bytes} [Wed Aug 24 08:48:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:48:02.246298 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d4e16f-6923-4fe5-8e3a-a365b5876e38 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:02.246844 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d4e16f-6923-4fe5-8e3a-a365b5876e38 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:02.246927 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d4e16f-6923-4fe5-8e3a-a365b5876e38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:02.342825 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-58d4e16f-6923-4fe5-8e3a-a365b5876e38 None None] MFA Rules not processed for user `cf0c99771661413aaa0f237af6b394ab`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:48:02.344456 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-58d4e16f-6923-4fe5-8e3a-a365b5876e38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:02.437239 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:02.479316 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 674/1353] 10.222.0.55 () {58 vars in 1018 bytes} [Wed Aug 24 08:48:02 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:48:04.456758 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc27506e-5387-44d1-9393-1f4415fa2517 None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:04.457306 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc27506e-5387-44d1-9393-1f4415fa2517 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:04.457399 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc27506e-5387-44d1-9393-1f4415fa2517 None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:04.459072 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 680/1354] 10.222.0.55 () {54 vars in 921 bytes} [Wed Aug 24 08:48:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:48:04.465312 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6078d7-d884-47ff-bba3-22a097256808 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:04.465847 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6078d7-d884-47ff-bba3-22a097256808 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:04.466266 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6078d7-d884-47ff-bba3-22a097256808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:04.553841 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-2a6078d7-d884-47ff-bba3-22a097256808 None None] MFA Rules not processed for user `cf0c99771661413aaa0f237af6b394ab`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:48:04.555010 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-2a6078d7-d884-47ff-bba3-22a097256808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:04.671193 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 675/1355] 10.222.0.55 () {58 vars in 1018 bytes} [Wed Aug 24 08:48:04 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:48:07.637400 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce268abd-6d01-4f05-a24d-0d2862cf5964 None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:07.638211 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce268abd-6d01-4f05-a24d-0d2862cf5964 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:07.638211 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce268abd-6d01-4f05-a24d-0d2862cf5964 None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:07.639340 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 681/1356] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:48:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:48:07.645624 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44d53ef-3955-4813-be68-7a11d417cee2 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:07.646161 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44d53ef-3955-4813-be68-7a11d417cee2 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:07.648823 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44d53ef-3955-4813-be68-7a11d417cee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:07.694575 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:07.745798 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-d44d53ef-3955-4813-be68-7a11d417cee2 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:48:07.746298 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-d44d53ef-3955-4813-be68-7a11d417cee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:07.889343 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 676/1357] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:48:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:48:08.234198 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca18df8-fd46-426f-ac8b-0bf2a043c91e None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:08.235324 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca18df8-fd46-426f-ac8b-0bf2a043c91e None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:08.235781 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca18df8-fd46-426f-ac8b-0bf2a043c91e None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:08.237394 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 682/1358] 10.222.0.55 () {54 vars in 962 bytes} [Wed Aug 24 08:48:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:48:08.247701 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f81c74-9811-43c3-9430-a16a31f1a09c None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:08.248523 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f81c74-9811-43c3-9430-a16a31f1a09c None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:08.249447 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f81c74-9811-43c3-9430-a16a31f1a09c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:08.361529 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-58f81c74-9811-43c3-9430-a16a31f1a09c None None] MFA Rules not processed for user `f17da357e555420799ee681bffbcc268`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:48:08.363056 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-58f81c74-9811-43c3-9430-a16a31f1a09c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:08.421396 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:08.494810 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 677/1359] 10.222.0.55 () {58 vars in 1059 bytes} [Wed Aug 24 08:48:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:48:08.503839 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca18df8-fd46-426f-ac8b-0bf2a043c91e None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:48:08.505603 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-2ca18df8-fd46-426f-ac8b-0bf2a043c91e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:08.684995 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca18df8-fd46-426f-ac8b-0bf2a043c91e None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:48:08.773833 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:08.818719 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb90fb2f-024a-4899-b171-ad40a5e87e40 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17da357e555420799ee681bffbcc268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:48:08.819689 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb90fb2f-024a-4899-b171-ad40a5e87e40 service nova] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:08.819984 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb90fb2f-024a-4899-b171-ad40a5e87e40 service nova] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:08.820269 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb90fb2f-024a-4899-b171-ad40a5e87e40 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:08.821916 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-eb90fb2f-024a-4899-b171-ad40a5e87e40 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:08.944444 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb90fb2f-024a-4899-b171-ad40a5e87e40 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:48:08.947624 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb90fb2f-024a-4899-b171-ad40a5e87e40 service nova] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:48:08.982204 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 683/1360] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:48:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:48:11.290872 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1860d61c-914f-4f6d-87fc-3824b100a38b None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:11.291379 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1860d61c-914f-4f6d-87fc-3824b100a38b None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:11.291379 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1860d61c-914f-4f6d-87fc-3824b100a38b None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:11.293424 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 678/1361] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:48:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:48:11.298923 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5311ca-c07f-48c5-8e23-c8b138c8a384 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:11.299411 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5311ca-c07f-48c5-8e23-c8b138c8a384 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:11.299494 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5311ca-c07f-48c5-8e23-c8b138c8a384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:11.376295 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-8d5311ca-c07f-48c5-8e23-c8b138c8a384 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:48:11.377463 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-8d5311ca-c07f-48c5-8e23-c8b138c8a384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:11.422546 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:11.504633 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 684/1362] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:48:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:48:11.913525 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1860d61c-914f-4f6d-87fc-3824b100a38b None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:48:11.915380 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-1860d61c-914f-4f6d-87fc-3824b100a38b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:12.081839 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1860d61c-914f-4f6d-87fc-3824b100a38b None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:48:12.093224 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:12.223560 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347a2c17-e05e-4769-b86e-f7b05ed8e11e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17da357e555420799ee681bffbcc268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:48:12.224170 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347a2c17-e05e-4769-b86e-f7b05ed8e11e service nova] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:12.224449 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347a2c17-e05e-4769-b86e-f7b05ed8e11e service nova] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:12.224755 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347a2c17-e05e-4769-b86e-f7b05ed8e11e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:12.226377 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-347a2c17-e05e-4769-b86e-f7b05ed8e11e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:12.327482 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347a2c17-e05e-4769-b86e-f7b05ed8e11e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:48:12.329231 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347a2c17-e05e-4769-b86e-f7b05ed8e11e service nova] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:48:12.350578 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 679/1363] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:48:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:48:14.729100 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8885f6aa-67e0-4550-ad25-de54e86f7afc None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:14.730297 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8885f6aa-67e0-4550-ad25-de54e86f7afc None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:14.730297 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8885f6aa-67e0-4550-ad25-de54e86f7afc None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:14.731546 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 685/1364] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:48:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:48:14.737392 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1341b30-0b66-4418-882c-723bdfe22319 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:14.737868 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1341b30-0b66-4418-882c-723bdfe22319 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:14.738005 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1341b30-0b66-4418-882c-723bdfe22319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:14.755666 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:14.815235 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-d1341b30-0b66-4418-882c-723bdfe22319 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:48:14.816504 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-d1341b30-0b66-4418-882c-723bdfe22319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:14.928866 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 680/1365] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:48:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:48:15.230189 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe151e7b-c1b0-45b8-818a-04817c71800f None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:15.230516 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe151e7b-c1b0-45b8-818a-04817c71800f None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:15.230781 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe151e7b-c1b0-45b8-818a-04817c71800f None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:15.232950 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 686/1366] 10.222.0.55 () {54 vars in 962 bytes} [Wed Aug 24 08:48:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:48:15.238606 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b7f9a-ee28-4e43-be8e-dc2562c24096 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:15.238894 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b7f9a-ee28-4e43-be8e-dc2562c24096 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:15.239159 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b7f9a-ee28-4e43-be8e-dc2562c24096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:15.332942 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-769b7f9a-ee28-4e43-be8e-dc2562c24096 None None] MFA Rules not processed for user `f17da357e555420799ee681bffbcc268`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:48:15.334198 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-769b7f9a-ee28-4e43-be8e-dc2562c24096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:15.374134 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:15.502752 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 681/1367] 10.222.0.55 () {58 vars in 1059 bytes} [Wed Aug 24 08:48:15 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:48:15.513746 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe151e7b-c1b0-45b8-818a-04817c71800f None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:48:15.515305 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-fe151e7b-c1b0-45b8-818a-04817c71800f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:15.701358 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe151e7b-c1b0-45b8-818a-04817c71800f None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:48:15.724801 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:15.829267 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff9f62a-a73c-42ba-adf1-e889753be390 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17da357e555420799ee681bffbcc268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:48:15.829989 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff9f62a-a73c-42ba-adf1-e889753be390 service nova] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:15.830249 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff9f62a-a73c-42ba-adf1-e889753be390 service nova] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:15.830420 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff9f62a-a73c-42ba-adf1-e889753be390 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:15.832060 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-5ff9f62a-a73c-42ba-adf1-e889753be390 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:15.930706 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff9f62a-a73c-42ba-adf1-e889753be390 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:48:15.933115 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff9f62a-a73c-42ba-adf1-e889753be390 service nova] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:48:15.952882 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 687/1368] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:48:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:48:16.273995 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd59cc9f-ba62-4124-a95b-07e8f6b3030f None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:16.274573 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd59cc9f-ba62-4124-a95b-07e8f6b3030f None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:16.274573 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd59cc9f-ba62-4124-a95b-07e8f6b3030f None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:16.280616 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 682/1369] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:48:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:48:16.283387 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb47f1b-d022-4040-8606-97be7cc519da None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:16.283728 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb47f1b-d022-4040-8606-97be7cc519da None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:16.284015 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb47f1b-d022-4040-8606-97be7cc519da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:16.324804 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:16.373762 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-3eb47f1b-d022-4040-8606-97be7cc519da None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:48:16.374981 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-3eb47f1b-d022-4040-8606-97be7cc519da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:16.483566 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 688/1370] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:48:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:48:17.097661 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a5326f-c186-4c67-98eb-558472775957 None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:17.098572 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a5326f-c186-4c67-98eb-558472775957 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:17.098572 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a5326f-c186-4c67-98eb-558472775957 None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:17.099801 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 683/1371] 10.222.0.55 () {54 vars in 964 bytes} [Wed Aug 24 08:48:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:48:17.105782 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1d59e6-21ef-4f43-9939-6603d9b97796 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:17.106306 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1d59e6-21ef-4f43-9939-6603d9b97796 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:17.106452 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1d59e6-21ef-4f43-9939-6603d9b97796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:17.195279 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-2a1d59e6-21ef-4f43-9939-6603d9b97796 None None] MFA Rules not processed for user `fbbb9ac667f04c35a63b2689df18efed`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:48:17.196669 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-2a1d59e6-21ef-4f43-9939-6603d9b97796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:17.251121 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:17.284859 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 689/1372] 10.222.0.55 () {58 vars in 1061 bytes} [Wed Aug 24 08:48:17 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:48:17.292412 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a5326f-c186-4c67-98eb-558472775957 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:48:17.293985 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-89a5326f-c186-4c67-98eb-558472775957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:17.408982 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a5326f-c186-4c67-98eb-558472775957 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:48:17.451680 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:17.494260 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97db72af-f48c-4fa2-893b-f642fb4e89df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbb9ac667f04c35a63b2689df18efed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:48:17.494260 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97db72af-f48c-4fa2-893b-f642fb4e89df service cinder] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:17.494260 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97db72af-f48c-4fa2-893b-f642fb4e89df service cinder] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:17.494260 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97db72af-f48c-4fa2-893b-f642fb4e89df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:17.495922 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-97db72af-f48c-4fa2-893b-f642fb4e89df service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:17.612226 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97db72af-f48c-4fa2-893b-f642fb4e89df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:48:17.613687 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97db72af-f48c-4fa2-893b-f642fb4e89df service cinder] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:48:17.634524 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 684/1373] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:48:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:48:18.152326 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb44491-9f08-470b-808e-a36eb4ec43a6 None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:18.153003 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb44491-9f08-470b-808e-a36eb4ec43a6 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:18.153003 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb44491-9f08-470b-808e-a36eb4ec43a6 None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:18.154249 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 690/1374] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:48:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:48:18.160010 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d058dc55-3bf1-4496-9a00-edeceaea510d None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:18.160319 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d058dc55-3bf1-4496-9a00-edeceaea510d None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:18.162171 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d058dc55-3bf1-4496-9a00-edeceaea510d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:18.236703 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-d058dc55-3bf1-4496-9a00-edeceaea510d None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:48:18.238025 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-d058dc55-3bf1-4496-9a00-edeceaea510d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:18.261834 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:18.344310 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 685/1375] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:48:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:48:18.637750 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb44491-9f08-470b-808e-a36eb4ec43a6 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:48:18.639380 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-bdb44491-9f08-470b-808e-a36eb4ec43a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:18.792181 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb44491-9f08-470b-808e-a36eb4ec43a6 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:48:18.847689 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:18.915800 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b96bcc-8cee-4338-ae6c-59838872b08a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17da357e555420799ee681bffbcc268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:48:18.916190 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b96bcc-8cee-4338-ae6c-59838872b08a service nova] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:18.916466 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b96bcc-8cee-4338-ae6c-59838872b08a service nova] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:18.917293 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b96bcc-8cee-4338-ae6c-59838872b08a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:18.918990 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-29b96bcc-8cee-4338-ae6c-59838872b08a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:19.018658 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b96bcc-8cee-4338-ae6c-59838872b08a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:48:19.020272 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b96bcc-8cee-4338-ae6c-59838872b08a service nova] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:48:19.043497 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 691/1376] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:48:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:48:21.238130 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7bed5f-ea84-4e2a-8ac9-2cb7a6383be5 None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:21.238648 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7bed5f-ea84-4e2a-8ac9-2cb7a6383be5 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:21.238843 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7bed5f-ea84-4e2a-8ac9-2cb7a6383be5 None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:21.240444 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 686/1377] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:48:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:48:21.247081 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97a25c9-4439-4511-8318-e2de25790cc3 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:21.247756 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97a25c9-4439-4511-8318-e2de25790cc3 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:21.247756 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97a25c9-4439-4511-8318-e2de25790cc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:21.333606 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-c97a25c9-4439-4511-8318-e2de25790cc3 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:48:21.334910 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-c97a25c9-4439-4511-8318-e2de25790cc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:21.342950 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:21.482798 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 692/1378] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:48:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:48:21.802151 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7bed5f-ea84-4e2a-8ac9-2cb7a6383be5 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:48:21.803802 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-2b7bed5f-ea84-4e2a-8ac9-2cb7a6383be5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:21.970493 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7bed5f-ea84-4e2a-8ac9-2cb7a6383be5 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:48:21.984769 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:22.101128 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2201b78-f565-4096-8d02-a708a02ef4d5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17da357e555420799ee681bffbcc268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:48:22.101843 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2201b78-f565-4096-8d02-a708a02ef4d5 service nova] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:22.102196 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2201b78-f565-4096-8d02-a708a02ef4d5 service nova] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:22.102539 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2201b78-f565-4096-8d02-a708a02ef4d5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:22.104857 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-e2201b78-f565-4096-8d02-a708a02ef4d5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:22.208410 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2201b78-f565-4096-8d02-a708a02ef4d5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:48:22.210496 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2201b78-f565-4096-8d02-a708a02ef4d5 service nova] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:48:22.232587 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 687/1379] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:48:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:48:24.554026 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6f5fec-4309-4870-9ab7-21fe22b8c16f None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:24.554026 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6f5fec-4309-4870-9ab7-21fe22b8c16f None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:24.554026 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6f5fec-4309-4870-9ab7-21fe22b8c16f None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:24.554026 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 693/1380] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:48:24 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:48:24.554993 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437815a0-83dc-46c2-895c-634acceaba10 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:24.554993 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437815a0-83dc-46c2-895c-634acceaba10 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:24.554993 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437815a0-83dc-46c2-895c-634acceaba10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:24.574468 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:24.654842 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-437815a0-83dc-46c2-895c-634acceaba10 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:48:24.656608 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-437815a0-83dc-46c2-895c-634acceaba10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:24.801787 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 688/1381] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:48:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:48:25.105881 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6f5fec-4309-4870-9ab7-21fe22b8c16f None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:48:25.107416 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-0b6f5fec-4309-4870-9ab7-21fe22b8c16f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:25.263125 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:25.277733 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6f5fec-4309-4870-9ab7-21fe22b8c16f None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:48:25.405402 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4d1ff5-1c61-4adf-97cc-386bb583d32a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17da357e555420799ee681bffbcc268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:48:25.405921 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4d1ff5-1c61-4adf-97cc-386bb583d32a service nova] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:25.406197 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4d1ff5-1c61-4adf-97cc-386bb583d32a service nova] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:25.406467 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4d1ff5-1c61-4adf-97cc-386bb583d32a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:25.408068 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-eb4d1ff5-1c61-4adf-97cc-386bb583d32a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:25.507114 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4d1ff5-1c61-4adf-97cc-386bb583d32a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:48:25.508952 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4d1ff5-1c61-4adf-97cc-386bb583d32a service nova] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:48:25.521475 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:25.530791 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 694/1382] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:48:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:48:27.736525 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0a94ca-cc02-498f-b282-90d2d6c48481 None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:27.737729 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0a94ca-cc02-498f-b282-90d2d6c48481 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:27.737729 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0a94ca-cc02-498f-b282-90d2d6c48481 None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:27.738456 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 689/1383] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:48:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:48:27.744350 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517787f3-5be4-4ab8-bb66-cd9dcbd19101 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:27.745561 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517787f3-5be4-4ab8-bb66-cd9dcbd19101 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:27.745561 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517787f3-5be4-4ab8-bb66-cd9dcbd19101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:27.839590 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-517787f3-5be4-4ab8-bb66-cd9dcbd19101 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:48:27.841400 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-517787f3-5be4-4ab8-bb66-cd9dcbd19101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:27.967254 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 695/1384] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:48:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:48:28.318970 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0a94ca-cc02-498f-b282-90d2d6c48481 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:48:28.320522 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-3a0a94ca-cc02-498f-b282-90d2d6c48481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:28.432980 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:28.519409 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0a94ca-cc02-498f-b282-90d2d6c48481 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:48:28.654648 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17555124-742e-468e-ba64-e40e7cc49de6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17da357e555420799ee681bffbcc268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:48:28.655273 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17555124-742e-468e-ba64-e40e7cc49de6 service nova] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:28.655573 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17555124-742e-468e-ba64-e40e7cc49de6 service nova] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:28.655891 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17555124-742e-468e-ba64-e40e7cc49de6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:28.657684 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-17555124-742e-468e-ba64-e40e7cc49de6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:28.726058 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:28.794636 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17555124-742e-468e-ba64-e40e7cc49de6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:48:28.796735 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17555124-742e-468e-ba64-e40e7cc49de6 service nova] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:48:28.820189 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 690/1385] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:48:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 502 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:48:31.175998 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4f421c-fe5e-4c97-a496-3e3d32eddc00 None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:31.176618 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4f421c-fe5e-4c97-a496-3e3d32eddc00 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:31.176742 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4f421c-fe5e-4c97-a496-3e3d32eddc00 None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:31.178338 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 696/1386] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:48:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:48:31.184900 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1da27ae-9274-4f46-8398-2734d6d730a6 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:31.185435 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1da27ae-9274-4f46-8398-2734d6d730a6 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:31.185556 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1da27ae-9274-4f46-8398-2734d6d730a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:31.281020 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-e1da27ae-9274-4f46-8398-2734d6d730a6 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:48:31.282287 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-e1da27ae-9274-4f46-8398-2734d6d730a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:31.409441 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 691/1387] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:48:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:48:31.833885 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4f421c-fe5e-4c97-a496-3e3d32eddc00 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:48:31.835986 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-0b4f421c-fe5e-4c97-a496-3e3d32eddc00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:31.929858 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:32.020302 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4f421c-fe5e-4c97-a496-3e3d32eddc00 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:48:32.155100 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2562ca96-3b2b-43d7-8fa7-01213a73577a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17da357e555420799ee681bffbcc268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:48:32.155515 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2562ca96-3b2b-43d7-8fa7-01213a73577a service nova] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:32.155799 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2562ca96-3b2b-43d7-8fa7-01213a73577a service nova] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:32.156084 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2562ca96-3b2b-43d7-8fa7-01213a73577a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:32.157757 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-2562ca96-3b2b-43d7-8fa7-01213a73577a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:32.215314 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:32.279817 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2562ca96-3b2b-43d7-8fa7-01213a73577a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:48:32.281536 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2562ca96-3b2b-43d7-8fa7-01213a73577a service nova] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:48:32.302785 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 697/1388] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:48:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:48:34.042208 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defbf170-f625-41cc-94e5-6fa0da55f2c4 None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:34.042775 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defbf170-f625-41cc-94e5-6fa0da55f2c4 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:34.042775 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defbf170-f625-41cc-94e5-6fa0da55f2c4 None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:34.044203 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 692/1389] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:48:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:48:34.050679 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c6d139-0559-4b82-96eb-d131f3cbe934 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:34.051956 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c6d139-0559-4b82-96eb-d131f3cbe934 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:34.051956 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c6d139-0559-4b82-96eb-d131f3cbe934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:34.127845 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-c9c6d139-0559-4b82-96eb-d131f3cbe934 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:48:34.129102 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-c9c6d139-0559-4b82-96eb-d131f3cbe934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:34.242036 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 698/1390] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:48:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:48:34.471103 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941fef28-c267-4dd7-9350-de2ab08bbbf9 None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:34.471384 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941fef28-c267-4dd7-9350-de2ab08bbbf9 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:34.471654 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941fef28-c267-4dd7-9350-de2ab08bbbf9 None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:34.473066 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 693/1391] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:48:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:48:34.478732 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9b2ff7-1e8d-4cf7-a1d4-cb4d97106f42 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:34.479079 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9b2ff7-1e8d-4cf7-a1d4-cb4d97106f42 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:34.479351 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9b2ff7-1e8d-4cf7-a1d4-cb4d97106f42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:34.495943 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:34.553549 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-eb9b2ff7-1e8d-4cf7-a1d4-cb4d97106f42 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:48:34.554794 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-eb9b2ff7-1e8d-4cf7-a1d4-cb4d97106f42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:34.564848 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951c527e-9fd2-4b9e-8811-daad9a414beb None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:34.565173 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951c527e-9fd2-4b9e-8811-daad9a414beb None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:34.565449 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951c527e-9fd2-4b9e-8811-daad9a414beb None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:34.566830 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 694/1392] 10.222.0.55 () {54 vars in 967 bytes} [Wed Aug 24 08:48:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:48:34.571959 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebaa1ce-06ae-4b28-a50a-96ee1933d68d None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:34.576260 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebaa1ce-06ae-4b28-a50a-96ee1933d68d None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:34.576260 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebaa1ce-06ae-4b28-a50a-96ee1933d68d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:34.607515 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:34.654852 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-aebaa1ce-06ae-4b28-a50a-96ee1933d68d None None] MFA Rules not processed for user `5512043ca9a7421b821741ca88d50b07`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:48:34.656076 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-aebaa1ce-06ae-4b28-a50a-96ee1933d68d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:34.673547 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 699/1393] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:48:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:48:34.734075 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 695/1394] 10.222.0.55 () {58 vars in 1064 bytes} [Wed Aug 24 08:48:34 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:48:34.741760 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9b2ff7-1e8d-4cf7-a1d4-cb4d97106f42 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:48:34.742969 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-eb9b2ff7-1e8d-4cf7-a1d4-cb4d97106f42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:34.840165 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9b2ff7-1e8d-4cf7-a1d4-cb4d97106f42 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:48:34.848329 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:34.926787 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1211ae-54cd-4b0c-8c78-aca589b7bcb4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5512043ca9a7421b821741ca88d50b07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:48:34.927415 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1211ae-54cd-4b0c-8c78-aca589b7bcb4 service glance] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:34.927691 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1211ae-54cd-4b0c-8c78-aca589b7bcb4 service glance] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:34.927994 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1211ae-54cd-4b0c-8c78-aca589b7bcb4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:34.929713 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-1b1211ae-54cd-4b0c-8c78-aca589b7bcb4 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:34.962843 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebaa1ce-06ae-4b28-a50a-96ee1933d68d None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:48:34.964098 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-aebaa1ce-06ae-4b28-a50a-96ee1933d68d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:35.073354 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1211ae-54cd-4b0c-8c78-aca589b7bcb4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:48:35.075019 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1211ae-54cd-4b0c-8c78-aca589b7bcb4 service glance] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:48:35.094433 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 700/1395] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:48:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:48:35.101368 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebaa1ce-06ae-4b28-a50a-96ee1933d68d None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:48:35.105729 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:35.223086 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0bc94b-e4f2-4c6d-a26f-dd4dadfd81dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17da357e555420799ee681bffbcc268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:48:35.223693 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0bc94b-e4f2-4c6d-a26f-dd4dadfd81dd service nova] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:35.223971 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0bc94b-e4f2-4c6d-a26f-dd4dadfd81dd service nova] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:35.224238 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0bc94b-e4f2-4c6d-a26f-dd4dadfd81dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:35.225874 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-4e0bc94b-e4f2-4c6d-a26f-dd4dadfd81dd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:35.321815 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0bc94b-e4f2-4c6d-a26f-dd4dadfd81dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:48:35.323506 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0bc94b-e4f2-4c6d-a26f-dd4dadfd81dd service nova] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:48:35.345877 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 696/1396] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:48:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:48:36.051933 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2503a3-6ec2-48e8-9b33-46f4f47d7859 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:36.052325 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2503a3-6ec2-48e8-9b33-46f4f47d7859 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:36.052627 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2503a3-6ec2-48e8-9b33-46f4f47d7859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:36.103841 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:36.128359 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-5f2503a3-6ec2-48e8-9b33-46f4f47d7859 None None] MFA Rules not processed for user `f652a368b1cc42ae8b6f6c7a6dff6f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:48:36.129636 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-5f2503a3-6ec2-48e8-9b33-46f4f47d7859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:36.237833 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 701/1397] 10.222.0.55 () {58 vars in 1009 bytes} [Wed Aug 24 08:48:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:48:36.285957 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e22886-0ea9-4364-99f9-a90cd861ef16 None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:36.286484 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e22886-0ea9-4364-99f9-a90cd861ef16 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:36.286484 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e22886-0ea9-4364-99f9-a90cd861ef16 None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:36.287869 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 697/1398] 10.222.0.55 () {54 vars in 943 bytes} [Wed Aug 24 08:48:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:48:36.294582 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77db924-7174-482b-92e6-16c2d75e1323 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:36.294867 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77db924-7174-482b-92e6-16c2d75e1323 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:36.295230 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77db924-7174-482b-92e6-16c2d75e1323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:36.393825 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-e77db924-7174-482b-92e6-16c2d75e1323 None None] MFA Rules not processed for user `a0ebb5ccabef45378f0c1da69721e051`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:48:36.395188 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-e77db924-7174-482b-92e6-16c2d75e1323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:36.469217 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:36.545820 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 702/1399] 10.222.0.55 () {58 vars in 1040 bytes} [Wed Aug 24 08:48:36 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:48:36.555393 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e22886-0ea9-4364-99f9-a90cd861ef16 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:48:36.557482 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-91e22886-0ea9-4364-99f9-a90cd861ef16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:36.567291 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:36.706318 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e22886-0ea9-4364-99f9-a90cd861ef16 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:48:36.823711 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:36.830685 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acc1b8a-7f33-4adb-b69c-ae93ede13896 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0ebb5ccabef45378f0c1da69721e051', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:48:36.831293 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acc1b8a-7f33-4adb-b69c-ae93ede13896 service swift] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:36.831567 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acc1b8a-7f33-4adb-b69c-ae93ede13896 service swift] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:36.831834 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acc1b8a-7f33-4adb-b69c-ae93ede13896 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:36.833516 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-9acc1b8a-7f33-4adb-b69c-ae93ede13896 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:36.925921 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acc1b8a-7f33-4adb-b69c-ae93ede13896 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:48:36.927910 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acc1b8a-7f33-4adb-b69c-ae93ede13896 service swift] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:48:36.938921 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 698/1400] 10.222.0.55 () {60 vars in 1384 bytes} [Wed Aug 24 08:48:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 08:48:37.605573 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036f68dc-165c-4330-a3fe-bb75312220e7 None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:37.606117 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036f68dc-165c-4330-a3fe-bb75312220e7 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:37.606419 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036f68dc-165c-4330-a3fe-bb75312220e7 None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:37.608030 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 703/1401] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:48:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:48:37.615519 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c84947-2c83-4eb3-8bad-fe50f2074c1d None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:37.616302 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c84947-2c83-4eb3-8bad-fe50f2074c1d None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:37.616302 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c84947-2c83-4eb3-8bad-fe50f2074c1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:37.736906 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-08c84947-2c83-4eb3-8bad-fe50f2074c1d None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:48:37.738350 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-08c84947-2c83-4eb3-8bad-fe50f2074c1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:37.912265 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:37.965325 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 699/1402] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:48:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:48:38.409214 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036f68dc-165c-4330-a3fe-bb75312220e7 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:48:38.411688 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-036f68dc-165c-4330-a3fe-bb75312220e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:38.714221 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036f68dc-165c-4330-a3fe-bb75312220e7 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:48:38.807500 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:38.972064 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c54da7-62a6-4c87-a545-6b10f5ab7554 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17da357e555420799ee681bffbcc268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:48:38.972891 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c54da7-62a6-4c87-a545-6b10f5ab7554 service nova] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:38.973367 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c54da7-62a6-4c87-a545-6b10f5ab7554 service nova] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:38.973781 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c54da7-62a6-4c87-a545-6b10f5ab7554 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:38.975754 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-b0c54da7-62a6-4c87-a545-6b10f5ab7554 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:39.136869 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c54da7-62a6-4c87-a545-6b10f5ab7554 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:48:39.139945 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c54da7-62a6-4c87-a545-6b10f5ab7554 service nova] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:48:39.176714 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 704/1403] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:48:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 768 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:48:42.717068 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc561feb-2a79-485e-aa79-dde93e7e8aac None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:42.717840 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc561feb-2a79-485e-aa79-dde93e7e8aac None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:42.718036 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc561feb-2a79-485e-aa79-dde93e7e8aac None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:42.720495 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 700/1404] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:48:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:48:42.728438 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbd977f-6892-433d-99e6-a6cbdc61cfd0 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:42.730098 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbd977f-6892-433d-99e6-a6cbdc61cfd0 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:42.731288 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbd977f-6892-433d-99e6-a6cbdc61cfd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:42.830731 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:42.852486 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-cbbd977f-6892-433d-99e6-a6cbdc61cfd0 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:48:42.855206 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-cbbd977f-6892-433d-99e6-a6cbdc61cfd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:43.045892 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 705/1405] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:48:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:48:43.588763 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc561feb-2a79-485e-aa79-dde93e7e8aac None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:48:43.590534 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-cc561feb-2a79-485e-aa79-dde93e7e8aac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:43.853188 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc561feb-2a79-485e-aa79-dde93e7e8aac None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:48:43.988866 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:44.074151 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fe3d27-ec3e-48a0-aa11-3ea4a1c39bfa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17da357e555420799ee681bffbcc268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:48:44.074768 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe3d27-ec3e-48a0-aa11-3ea4a1c39bfa service nova] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:44.075047 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe3d27-ec3e-48a0-aa11-3ea4a1c39bfa service nova] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:44.075316 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe3d27-ec3e-48a0-aa11-3ea4a1c39bfa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:44.077235 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-17fe3d27-ec3e-48a0-aa11-3ea4a1c39bfa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:44.196103 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fe3d27-ec3e-48a0-aa11-3ea4a1c39bfa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:48:44.198657 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fe3d27-ec3e-48a0-aa11-3ea4a1c39bfa service nova] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:48:44.225905 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 701/1406] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:48:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 638 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:48:46.468326 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99e9fd6-484f-4c26-a24a-dd8d48526e62 None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:46.469288 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99e9fd6-484f-4c26-a24a-dd8d48526e62 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:46.469823 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99e9fd6-484f-4c26-a24a-dd8d48526e62 None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:46.471179 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 706/1407] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:48:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:48:46.476980 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c79ce7-02d1-4463-b7b1-b248f6d4edc3 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:46.477519 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c79ce7-02d1-4463-b7b1-b248f6d4edc3 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:46.477619 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c79ce7-02d1-4463-b7b1-b248f6d4edc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:46.562330 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-d1c79ce7-02d1-4463-b7b1-b248f6d4edc3 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:48:46.563721 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-d1c79ce7-02d1-4463-b7b1-b248f6d4edc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:46.591890 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:46.729108 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 702/1408] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:48:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:48:47.117609 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99e9fd6-484f-4c26-a24a-dd8d48526e62 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:48:47.119981 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-c99e9fd6-484f-4c26-a24a-dd8d48526e62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:47.262788 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:47.298549 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99e9fd6-484f-4c26-a24a-dd8d48526e62 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:48:47.442145 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee594bfd-f648-4cdb-862f-2c034a9f34cc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17da357e555420799ee681bffbcc268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:48:47.443258 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee594bfd-f648-4cdb-862f-2c034a9f34cc service nova] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:47.443763 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee594bfd-f648-4cdb-862f-2c034a9f34cc service nova] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:47.444255 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee594bfd-f648-4cdb-862f-2c034a9f34cc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:47.446918 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-ee594bfd-f648-4cdb-862f-2c034a9f34cc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:47.557077 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:47.561036 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee594bfd-f648-4cdb-862f-2c034a9f34cc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:48:47.562654 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee594bfd-f648-4cdb-862f-2c034a9f34cc service nova] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:48:47.582410 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 707/1409] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:48:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:48:56.714363 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=33 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:57.589522 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=7 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:01.789766 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ce7468-3163-475f-a4ce-ea909ec81f5d None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:01.790382 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ce7468-3163-475f-a4ce-ea909ec81f5d None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:01.790382 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ce7468-3163-475f-a4ce-ea909ec81f5d None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:01.792277 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 703/1410] 10.222.0.55 () {54 vars in 918 bytes} [Wed Aug 24 08:49:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:49:01.797641 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23c71b3-6e6a-4805-bfa9-8074a9fd3bdc None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:01.798117 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23c71b3-6e6a-4805-bfa9-8074a9fd3bdc None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:01.798341 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23c71b3-6e6a-4805-bfa9-8074a9fd3bdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:01.886945 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-f23c71b3-6e6a-4805-bfa9-8074a9fd3bdc None None] MFA Rules not processed for user `cf0c99771661413aaa0f237af6b394ab`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:49:01.888218 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-f23c71b3-6e6a-4805-bfa9-8074a9fd3bdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:02.008555 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 708/1411] 10.222.0.55 () {58 vars in 1015 bytes} [Wed Aug 24 08:49:01 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:49:02.024904 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b797bd0-9e68-4d4b-aa16-4cef81b4ff6e None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:02.025187 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b797bd0-9e68-4d4b-aa16-4cef81b4ff6e None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:02.025486 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b797bd0-9e68-4d4b-aa16-4cef81b4ff6e None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:02.026856 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 704/1412] 10.222.0.55 () {54 vars in 918 bytes} [Wed Aug 24 08:49:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:49:02.032650 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5bfb2f-a378-458b-a14d-fd9c0d5a8d03 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:02.032932 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5bfb2f-a378-458b-a14d-fd9c0d5a8d03 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:02.033207 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5bfb2f-a378-458b-a14d-fd9c0d5a8d03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:02.115670 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:02.120715 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-2d5bfb2f-a378-458b-a14d-fd9c0d5a8d03 None None] MFA Rules not processed for user `f17da357e555420799ee681bffbcc268`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:49:02.121966 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-2d5bfb2f-a378-458b-a14d-fd9c0d5a8d03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:02.255238 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 709/1413] 10.222.0.55 () {58 vars in 1015 bytes} [Wed Aug 24 08:49:02 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:49:04.697148 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654b13c4-a188-49a0-8894-7beff7ccd535 None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:04.697717 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654b13c4-a188-49a0-8894-7beff7ccd535 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:04.697717 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654b13c4-a188-49a0-8894-7beff7ccd535 None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:04.699021 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 705/1414] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 08:49:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:49:04.704816 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2254947-17cb-4383-8f33-ed7230846d0f None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:04.705315 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2254947-17cb-4383-8f33-ed7230846d0f None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:04.705701 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2254947-17cb-4383-8f33-ed7230846d0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:04.784364 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-c2254947-17cb-4383-8f33-ed7230846d0f None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:49:04.785724 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-c2254947-17cb-4383-8f33-ed7230846d0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:04.851450 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:04.896097 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 710/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 08:49:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:49:05.187899 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654b13c4-a188-49a0-8894-7beff7ccd535 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:49:05.189529 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-654b13c4-a188-49a0-8894-7beff7ccd535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:05.345087 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654b13c4-a188-49a0-8894-7beff7ccd535 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:49:05.435736 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:05.489744 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b74efaf-2a0b-428c-aa74-dcb656ab5f84 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17da357e555420799ee681bffbcc268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:49:05.490647 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b74efaf-2a0b-428c-aa74-dcb656ab5f84 service nova] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:05.491066 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b74efaf-2a0b-428c-aa74-dcb656ab5f84 service nova] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:05.491403 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b74efaf-2a0b-428c-aa74-dcb656ab5f84 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:05.494131 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-9b74efaf-2a0b-428c-aa74-dcb656ab5f84 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:05.595423 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b74efaf-2a0b-428c-aa74-dcb656ab5f84 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:49:05.597068 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b74efaf-2a0b-428c-aa74-dcb656ab5f84 service nova] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:49:05.616087 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 706/1416] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:49:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:49:06.543422 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9044b591-0b87-4115-8742-9637278f50a2 None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:06.544015 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9044b591-0b87-4115-8742-9637278f50a2 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:06.544015 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9044b591-0b87-4115-8742-9637278f50a2 None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:06.545361 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 711/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 08:49:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:49:06.550799 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d17ca8-47a8-4a3a-b67d-199c1eab3462 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:06.551276 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d17ca8-47a8-4a3a-b67d-199c1eab3462 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:06.551398 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d17ca8-47a8-4a3a-b67d-199c1eab3462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:06.634749 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-64d17ca8-47a8-4a3a-b67d-199c1eab3462 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:49:06.635986 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-64d17ca8-47a8-4a3a-b67d-199c1eab3462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:06.664500 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:06.741468 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 707/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 08:49:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:49:06.896907 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9044b591-0b87-4115-8742-9637278f50a2 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:49:06.898534 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-9044b591-0b87-4115-8742-9637278f50a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:07.069164 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9044b591-0b87-4115-8742-9637278f50a2 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:49:07.113841 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:07.195982 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bc9647-97d6-41f8-baa1-ab30df76c079 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17da357e555420799ee681bffbcc268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:49:07.197226 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bc9647-97d6-41f8-baa1-ab30df76c079 service nova] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:07.197507 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bc9647-97d6-41f8-baa1-ab30df76c079 service nova] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:07.197773 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bc9647-97d6-41f8-baa1-ab30df76c079 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:07.199558 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-c9bc9647-97d6-41f8-baa1-ab30df76c079 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:07.297630 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bc9647-97d6-41f8-baa1-ab30df76c079 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:49:07.299250 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bc9647-97d6-41f8-baa1-ab30df76c079 service nova] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:49:07.321018 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 712/1419] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:49:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:49:08.242910 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af630e95-4970-4e7a-a392-135434c84d1e None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:08.243615 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af630e95-4970-4e7a-a392-135434c84d1e None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:08.243902 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af630e95-4970-4e7a-a392-135434c84d1e None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:08.245314 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 708/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 08:49:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:49:08.251123 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c366c634-514a-4170-aedb-c384498cc721 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:08.251123 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c366c634-514a-4170-aedb-c384498cc721 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:08.251814 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c366c634-514a-4170-aedb-c384498cc721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:08.309748 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:08.335716 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-c366c634-514a-4170-aedb-c384498cc721 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:49:08.337619 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-c366c634-514a-4170-aedb-c384498cc721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:08.437311 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 713/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 08:49:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:49:08.594727 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af630e95-4970-4e7a-a392-135434c84d1e None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:49:08.596273 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-af630e95-4970-4e7a-a392-135434c84d1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:08.801718 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af630e95-4970-4e7a-a392-135434c84d1e None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:49:08.823291 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:08.942421 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af03a0c4-abfe-4065-ae65-0cfb5072b4fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17da357e555420799ee681bffbcc268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:49:08.942939 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af03a0c4-abfe-4065-ae65-0cfb5072b4fd service nova] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:08.943216 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af03a0c4-abfe-4065-ae65-0cfb5072b4fd service nova] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:08.943481 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af03a0c4-abfe-4065-ae65-0cfb5072b4fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:08.945114 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-af03a0c4-abfe-4065-ae65-0cfb5072b4fd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:09.036619 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af03a0c4-abfe-4065-ae65-0cfb5072b4fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:49:09.038228 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af03a0c4-abfe-4065-ae65-0cfb5072b4fd service nova] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:49:09.056622 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 709/1422] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:49:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:49:09.944002 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846f34dc-2535-4a75-9aaa-ba87b9ca1639 None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:09.944625 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846f34dc-2535-4a75-9aaa-ba87b9ca1639 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:09.944959 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846f34dc-2535-4a75-9aaa-ba87b9ca1639 None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:09.947621 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 714/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 08:49:09 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:49:09.952660 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81a0b57-a9d3-4e76-88c1-fbc2301dc828 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:09.953091 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81a0b57-a9d3-4e76-88c1-fbc2301dc828 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:09.953210 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81a0b57-a9d3-4e76-88c1-fbc2301dc828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:09.975192 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:10.036762 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-c81a0b57-a9d3-4e76-88c1-fbc2301dc828 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:49:10.037998 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-c81a0b57-a9d3-4e76-88c1-fbc2301dc828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:10.140335 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 710/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 08:49:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:49:10.299323 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846f34dc-2535-4a75-9aaa-ba87b9ca1639 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:49:10.301080 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-846f34dc-2535-4a75-9aaa-ba87b9ca1639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:10.436036 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:10.473569 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846f34dc-2535-4a75-9aaa-ba87b9ca1639 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:49:10.619200 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7f0389-c233-43f9-a70a-e204453f9f93 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17da357e555420799ee681bffbcc268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:49:10.619820 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7f0389-c233-43f9-a70a-e204453f9f93 service nova] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:10.620183 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7f0389-c233-43f9-a70a-e204453f9f93 service nova] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:10.620479 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7f0389-c233-43f9-a70a-e204453f9f93 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:10.622234 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-dc7f0389-c233-43f9-a70a-e204453f9f93 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:10.706370 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:10.720908 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7f0389-c233-43f9-a70a-e204453f9f93 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:49:10.722510 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7f0389-c233-43f9-a70a-e204453f9f93 service nova] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:49:10.742041 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 715/1425] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:49:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:49:11.621029 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0d5671-74b5-49d0-b029-f01e43652e15 None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:11.621719 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0d5671-74b5-49d0-b029-f01e43652e15 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:11.621719 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0d5671-74b5-49d0-b029-f01e43652e15 None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:11.623119 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 711/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 08:49:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:49:11.630974 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96924ff3-319b-4551-93d7-0963499f1614 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:11.631469 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96924ff3-319b-4551-93d7-0963499f1614 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:11.631572 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96924ff3-319b-4551-93d7-0963499f1614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:11.713291 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-96924ff3-319b-4551-93d7-0963499f1614 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:49:11.714672 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-96924ff3-319b-4551-93d7-0963499f1614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:11.814601 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 716/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 08:49:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:49:11.966831 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0d5671-74b5-49d0-b029-f01e43652e15 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:49:11.968260 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-dd0d5671-74b5-49d0-b029-f01e43652e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:12.060219 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:12.150983 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0d5671-74b5-49d0-b029-f01e43652e15 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:49:12.273507 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663d2d45-c39d-480f-953c-dff079a67752 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17da357e555420799ee681bffbcc268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:49:12.274027 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663d2d45-c39d-480f-953c-dff079a67752 service nova] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:12.274839 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663d2d45-c39d-480f-953c-dff079a67752 service nova] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:12.275115 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663d2d45-c39d-480f-953c-dff079a67752 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:12.276749 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-663d2d45-c39d-480f-953c-dff079a67752 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:12.326400 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:12.369265 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663d2d45-c39d-480f-953c-dff079a67752 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:49:12.370860 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663d2d45-c39d-480f-953c-dff079a67752 service nova] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:49:12.389243 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 712/1428] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:49:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:49:13.284161 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fd316f-17b0-42c1-b463-2a0efa88b1f5 None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:13.284761 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fd316f-17b0-42c1-b463-2a0efa88b1f5 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:13.284761 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fd316f-17b0-42c1-b463-2a0efa88b1f5 None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:13.286089 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 717/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 08:49:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:49:13.291926 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a236e2-c903-4329-b959-6b7215591475 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:13.292434 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a236e2-c903-4329-b959-6b7215591475 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:13.292572 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a236e2-c903-4329-b959-6b7215591475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:13.373915 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-35a236e2-c903-4329-b959-6b7215591475 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:49:13.375142 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-35a236e2-c903-4329-b959-6b7215591475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:13.477607 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 713/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 08:49:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:49:13.630488 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fd316f-17b0-42c1-b463-2a0efa88b1f5 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:49:13.632051 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-b0fd316f-17b0-42c1-b463-2a0efa88b1f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:13.688624 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:13.800365 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fd316f-17b0-42c1-b463-2a0efa88b1f5 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:49:13.922210 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912aa8c9-c4bd-4b52-820e-7232e15e076d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17da357e555420799ee681bffbcc268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:49:13.922822 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912aa8c9-c4bd-4b52-820e-7232e15e076d service nova] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:13.923107 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912aa8c9-c4bd-4b52-820e-7232e15e076d service nova] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:13.923376 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912aa8c9-c4bd-4b52-820e-7232e15e076d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:13.925029 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-912aa8c9-c4bd-4b52-820e-7232e15e076d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:13.932407 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:14.018602 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912aa8c9-c4bd-4b52-820e-7232e15e076d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:49:14.020492 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912aa8c9-c4bd-4b52-820e-7232e15e076d service nova] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:49:14.039663 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 718/1431] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:49:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:49:14.915986 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7919c3-a813-4885-a373-f98616759601 None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:14.916587 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7919c3-a813-4885-a373-f98616759601 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:14.916587 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7919c3-a813-4885-a373-f98616759601 None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:14.918056 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 714/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 08:49:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:49:14.923331 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f6ca26-9b39-4921-943d-cb75e9dbafbb None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:14.923788 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f6ca26-9b39-4921-943d-cb75e9dbafbb None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:14.923950 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f6ca26-9b39-4921-943d-cb75e9dbafbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:15.002593 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-d2f6ca26-9b39-4921-943d-cb75e9dbafbb None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:49:15.003844 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-d2f6ca26-9b39-4921-943d-cb75e9dbafbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:15.074527 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:15.105622 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 719/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 08:49:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:49:15.261513 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7919c3-a813-4885-a373-f98616759601 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:49:15.263069 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-ef7919c3-a813-4885-a373-f98616759601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:15.282556 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:15.417936 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7919c3-a813-4885-a373-f98616759601 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:49:15.541606 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:15.569883 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4ca6be-8d0a-456e-b85f-7b664e037fbb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17da357e555420799ee681bffbcc268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:49:15.570494 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4ca6be-8d0a-456e-b85f-7b664e037fbb service nova] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:15.570773 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4ca6be-8d0a-456e-b85f-7b664e037fbb service nova] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:15.571736 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4ca6be-8d0a-456e-b85f-7b664e037fbb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:15.573424 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-df4ca6be-8d0a-456e-b85f-7b664e037fbb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:15.671737 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4ca6be-8d0a-456e-b85f-7b664e037fbb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:49:15.673378 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4ca6be-8d0a-456e-b85f-7b664e037fbb service nova] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:49:15.691598 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 715/1434] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:49:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:49:16.594312 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00dfba2-3baf-4fc2-8412-b6cd435074ec None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:16.594938 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00dfba2-3baf-4fc2-8412-b6cd435074ec None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:16.594938 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00dfba2-3baf-4fc2-8412-b6cd435074ec None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:16.596197 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 720/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 08:49:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:49:16.601380 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05ec703-d181-41f9-8d77-37f426756766 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:16.601839 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05ec703-d181-41f9-8d77-37f426756766 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:16.601940 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05ec703-d181-41f9-8d77-37f426756766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:16.682786 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-d05ec703-d181-41f9-8d77-37f426756766 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:49:16.684016 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-d05ec703-d181-41f9-8d77-37f426756766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:16.734381 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:16.798065 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 716/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 08:49:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:49:16.954489 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00dfba2-3baf-4fc2-8412-b6cd435074ec None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:49:16.955943 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-b00dfba2-3baf-4fc2-8412-b6cd435074ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:17.111699 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00dfba2-3baf-4fc2-8412-b6cd435074ec None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:49:17.169959 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:17.231266 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef8fd47-c091-485e-beb3-36e60a0e4c90 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17da357e555420799ee681bffbcc268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:49:17.231787 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef8fd47-c091-485e-beb3-36e60a0e4c90 service nova] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:17.232063 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef8fd47-c091-485e-beb3-36e60a0e4c90 service nova] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:17.232332 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef8fd47-c091-485e-beb3-36e60a0e4c90 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:17.233980 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-5ef8fd47-c091-485e-beb3-36e60a0e4c90 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:17.336572 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef8fd47-c091-485e-beb3-36e60a0e4c90 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:49:17.338225 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef8fd47-c091-485e-beb3-36e60a0e4c90 service nova] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:49:17.359156 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 721/1437] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:49:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:49:18.269673 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274bd1f5-ccdc-4325-915d-de0f2f6ed775 None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:18.269673 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274bd1f5-ccdc-4325-915d-de0f2f6ed775 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:18.269673 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274bd1f5-ccdc-4325-915d-de0f2f6ed775 None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:18.269673 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 717/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 08:49:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:49:18.274969 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bf6400-862f-4484-afa6-e93a2d69cb0c None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:18.275639 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bf6400-862f-4484-afa6-e93a2d69cb0c None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:18.275639 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bf6400-862f-4484-afa6-e93a2d69cb0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:18.358538 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-10bf6400-862f-4484-afa6-e93a2d69cb0c None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:49:18.359784 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-10bf6400-862f-4484-afa6-e93a2d69cb0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:18.362061 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:18.463525 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 722/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 08:49:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:49:18.615249 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274bd1f5-ccdc-4325-915d-de0f2f6ed775 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:49:18.616923 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-274bd1f5-ccdc-4325-915d-de0f2f6ed775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:18.770629 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274bd1f5-ccdc-4325-915d-de0f2f6ed775 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:49:18.799074 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:18.890919 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef577ef-d8b3-4acc-8d72-ef71cadd79fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17da357e555420799ee681bffbcc268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:49:18.891472 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef577ef-d8b3-4acc-8d72-ef71cadd79fe service nova] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:18.891747 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef577ef-d8b3-4acc-8d72-ef71cadd79fe service nova] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:18.892014 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef577ef-d8b3-4acc-8d72-ef71cadd79fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:18.893629 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-3ef577ef-d8b3-4acc-8d72-ef71cadd79fe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:18.983540 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef577ef-d8b3-4acc-8d72-ef71cadd79fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:49:18.985171 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef577ef-d8b3-4acc-8d72-ef71cadd79fe service nova] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:49:19.003156 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 718/1440] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:49:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:49:20.864353 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f792cb8-f847-4fcf-8c28-1fd5fe83ae62 None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:20.864973 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f792cb8-f847-4fcf-8c28-1fd5fe83ae62 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:20.864973 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f792cb8-f847-4fcf-8c28-1fd5fe83ae62 None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:20.866349 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 723/1441] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 08:49:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:49:20.873420 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd23cd9-26b1-4e50-81b4-0194350b769a None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:20.874058 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd23cd9-26b1-4e50-81b4-0194350b769a None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:20.874058 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd23cd9-26b1-4e50-81b4-0194350b769a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:20.918187 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:20.944946 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-9fd23cd9-26b1-4e50-81b4-0194350b769a None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:49:20.946219 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-9fd23cd9-26b1-4e50-81b4-0194350b769a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:21.077620 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 719/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 08:49:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:49:21.363668 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f792cb8-f847-4fcf-8c28-1fd5fe83ae62 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:49:21.365260 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-3f792cb8-f847-4fcf-8c28-1fd5fe83ae62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:21.485821 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f792cb8-f847-4fcf-8c28-1fd5fe83ae62 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:49:21.522029 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:21.574227 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfc1374-70bf-4614-99d7-f18ac351f997 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:49:21.574970 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfc1374-70bf-4614-99d7-f18ac351f997 service neutron] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:21.575302 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfc1374-70bf-4614-99d7-f18ac351f997 service neutron] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:21.575609 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfc1374-70bf-4614-99d7-f18ac351f997 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:21.577597 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-abfc1374-70bf-4614-99d7-f18ac351f997 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:21.707218 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfc1374-70bf-4614-99d7-f18ac351f997 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:49:21.713756 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfc1374-70bf-4614-99d7-f18ac351f997 service neutron] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:49:21.738077 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 724/1443] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:49:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:49:23.871892 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f5d08f-9edd-4015-addc-30eaf1207491 None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:23.872781 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f5d08f-9edd-4015-addc-30eaf1207491 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:23.872949 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f5d08f-9edd-4015-addc-30eaf1207491 None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:23.874361 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 720/1444] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 08:49:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:49:23.880191 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2512ba-749a-46d4-a59e-d5c95d897708 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:23.880766 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2512ba-749a-46d4-a59e-d5c95d897708 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:23.880766 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2512ba-749a-46d4-a59e-d5c95d897708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:23.954457 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-2f2512ba-749a-46d4-a59e-d5c95d897708 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:49:23.955703 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-2f2512ba-749a-46d4-a59e-d5c95d897708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:23.968947 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:24.082344 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 725/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 08:49:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:49:24.367545 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f5d08f-9edd-4015-addc-30eaf1207491 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:49:24.370652 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-d4f5d08f-9edd-4015-addc-30eaf1207491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:24.487487 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f5d08f-9edd-4015-addc-30eaf1207491 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:49:24.501650 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:24.570163 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2e3078-ddc1-4452-a97d-8c8965881927 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:49:24.571279 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e3078-ddc1-4452-a97d-8c8965881927 service neutron] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:24.571560 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e3078-ddc1-4452-a97d-8c8965881927 service neutron] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:24.571829 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e3078-ddc1-4452-a97d-8c8965881927 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:24.573516 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-1e2e3078-ddc1-4452-a97d-8c8965881927 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:24.698042 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2e3078-ddc1-4452-a97d-8c8965881927 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:49:24.699471 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2e3078-ddc1-4452-a97d-8c8965881927 service neutron] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:49:24.719555 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 721/1446] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:49:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:49:27.073773 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d455148e-24d2-4414-b184-48f1e2aaacec None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:27.074586 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d455148e-24d2-4414-b184-48f1e2aaacec None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:27.074586 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d455148e-24d2-4414-b184-48f1e2aaacec None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:27.075691 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 726/1447] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 08:49:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:49:27.081370 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3b3cd7-f88f-408f-b581-29f41d13ed8a None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:27.081815 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3b3cd7-f88f-408f-b581-29f41d13ed8a None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:27.081915 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3b3cd7-f88f-408f-b581-29f41d13ed8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:27.133250 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:27.152036 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-dc3b3cd7-f88f-408f-b581-29f41d13ed8a None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:49:27.153314 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-dc3b3cd7-f88f-408f-b581-29f41d13ed8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:27.261988 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 722/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 08:49:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:49:27.545706 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d455148e-24d2-4414-b184-48f1e2aaacec None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:49:27.547324 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-d455148e-24d2-4414-b184-48f1e2aaacec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:27.662511 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d455148e-24d2-4414-b184-48f1e2aaacec None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:49:27.705831 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:27.741375 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8bea92-6b04-4704-98e2-fe5ebc027935 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:49:27.742000 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8bea92-6b04-4704-98e2-fe5ebc027935 service neutron] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:27.742522 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8bea92-6b04-4704-98e2-fe5ebc027935 service neutron] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:27.743349 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8bea92-6b04-4704-98e2-fe5ebc027935 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:27.745725 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-8d8bea92-6b04-4704-98e2-fe5ebc027935 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:27.858725 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8bea92-6b04-4704-98e2-fe5ebc027935 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:49:27.860267 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8bea92-6b04-4704-98e2-fe5ebc027935 service neutron] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:49:27.879769 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 727/1449] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:49:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:49:30.067286 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c38811-c311-4140-9615-3f9f7e0f6835 None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:30.068073 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c38811-c311-4140-9615-3f9f7e0f6835 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:30.068073 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c38811-c311-4140-9615-3f9f7e0f6835 None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:30.069210 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 723/1450] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 08:49:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:49:30.074901 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8eba82-fa2b-4bc5-8724-5a813091a44b None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:30.075423 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8eba82-fa2b-4bc5-8724-5a813091a44b None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:30.075423 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8eba82-fa2b-4bc5-8724-5a813091a44b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:30.146982 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-2a8eba82-fa2b-4bc5-8724-5a813091a44b None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:49:30.148249 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-2a8eba82-fa2b-4bc5-8724-5a813091a44b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:30.175367 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:30.264211 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 728/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 08:49:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:49:30.550814 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c38811-c311-4140-9615-3f9f7e0f6835 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:49:30.552405 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-a6c38811-c311-4140-9615-3f9f7e0f6835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:30.666282 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c38811-c311-4140-9615-3f9f7e0f6835 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:49:30.688419 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:30.744203 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2530b7-9b18-450a-b6ac-4b53fdfe2102 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:49:30.744834 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2530b7-9b18-450a-b6ac-4b53fdfe2102 service neutron] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:30.745116 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2530b7-9b18-450a-b6ac-4b53fdfe2102 service neutron] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:30.745384 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2530b7-9b18-450a-b6ac-4b53fdfe2102 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:30.746982 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-3c2530b7-9b18-450a-b6ac-4b53fdfe2102 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:30.867563 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2530b7-9b18-450a-b6ac-4b53fdfe2102 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:49:30.869039 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2530b7-9b18-450a-b6ac-4b53fdfe2102 service neutron] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:49:30.887856 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 724/1452] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:49:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:49:33.092398 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f13b67b-5217-422a-8f22-785e58e375da None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:33.093089 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f13b67b-5217-422a-8f22-785e58e375da None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:33.093089 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f13b67b-5217-422a-8f22-785e58e375da None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:33.094344 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 729/1453] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 08:49:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:49:33.100205 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb43ae36-1b3c-45de-a86c-0fae9f404609 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:33.101155 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb43ae36-1b3c-45de-a86c-0fae9f404609 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:33.101155 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb43ae36-1b3c-45de-a86c-0fae9f404609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:33.168838 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:33.171560 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-cb43ae36-1b3c-45de-a86c-0fae9f404609 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:49:33.172829 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-cb43ae36-1b3c-45de-a86c-0fae9f404609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:33.277375 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 725/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 08:49:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:49:33.559092 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f13b67b-5217-422a-8f22-785e58e375da None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:49:33.560767 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-6f13b67b-5217-422a-8f22-785e58e375da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:33.685761 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f13b67b-5217-422a-8f22-785e58e375da None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:49:33.740166 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:33.766377 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2226bfa-0612-432d-ab84-1cf8bf04b1d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:49:33.767068 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2226bfa-0612-432d-ab84-1cf8bf04b1d3 service neutron] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:33.767348 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2226bfa-0612-432d-ab84-1cf8bf04b1d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:33.767619 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2226bfa-0612-432d-ab84-1cf8bf04b1d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:33.769518 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-f2226bfa-0612-432d-ab84-1cf8bf04b1d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:33.884968 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2226bfa-0612-432d-ab84-1cf8bf04b1d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:49:33.886431 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2226bfa-0612-432d-ab84-1cf8bf04b1d3 service neutron] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:49:33.906349 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 730/1455] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:49:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:49:35.913100 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691268b3-2470-48be-acc6-86f8b53f95bc None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:35.913677 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691268b3-2470-48be-acc6-86f8b53f95bc None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:35.913677 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691268b3-2470-48be-acc6-86f8b53f95bc None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:35.915077 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 726/1456] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 08:49:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:49:35.921456 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da91e857-6e82-4a55-914f-5713936db0f1 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:35.921456 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da91e857-6e82-4a55-914f-5713936db0f1 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:35.922070 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da91e857-6e82-4a55-914f-5713936db0f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:35.995513 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-da91e857-6e82-4a55-914f-5713936db0f1 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:49:35.996777 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-da91e857-6e82-4a55-914f-5713936db0f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:36.027549 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:36.102185 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 731/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 08:49:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:49:36.383855 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691268b3-2470-48be-acc6-86f8b53f95bc None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:49:36.385463 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-691268b3-2470-48be-acc6-86f8b53f95bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:36.490990 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691268b3-2470-48be-acc6-86f8b53f95bc None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:49:36.520702 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:36.566858 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ee1ffe-b2a7-4edf-9bd8-e298549cdbf7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:49:36.567464 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ee1ffe-b2a7-4edf-9bd8-e298549cdbf7 service neutron] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:36.567744 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ee1ffe-b2a7-4edf-9bd8-e298549cdbf7 service neutron] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:36.568013 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ee1ffe-b2a7-4edf-9bd8-e298549cdbf7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:36.569671 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-36ee1ffe-b2a7-4edf-9bd8-e298549cdbf7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:36.800789 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ee1ffe-b2a7-4edf-9bd8-e298549cdbf7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:49:36.802127 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ee1ffe-b2a7-4edf-9bd8-e298549cdbf7 service neutron] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:49:36.820876 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 727/1458] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:49:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:49:38.780775 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6a574d-9ca5-4830-afd9-5548cd469863 None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:38.781619 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6a574d-9ca5-4830-afd9-5548cd469863 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:38.781619 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6a574d-9ca5-4830-afd9-5548cd469863 None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:38.782726 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 732/1459] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 08:49:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:49:38.788146 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f97d9e5-1cdf-44a5-9f55-14967619926a None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:38.788673 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f97d9e5-1cdf-44a5-9f55-14967619926a None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:38.788797 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f97d9e5-1cdf-44a5-9f55-14967619926a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:38.868345 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-8f97d9e5-1cdf-44a5-9f55-14967619926a None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:49:38.869652 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-8f97d9e5-1cdf-44a5-9f55-14967619926a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:38.879762 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:38.974938 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 728/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 08:49:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:49:39.257931 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6a574d-9ca5-4830-afd9-5548cd469863 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:49:39.259483 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-ed6a574d-9ca5-4830-afd9-5548cd469863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:39.373038 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6a574d-9ca5-4830-afd9-5548cd469863 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:49:39.434499 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:39.457666 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd96d304-5234-4b7c-ad48-4ab9349430ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:49:39.458192 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd96d304-5234-4b7c-ad48-4ab9349430ed service neutron] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:39.458467 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd96d304-5234-4b7c-ad48-4ab9349430ed service neutron] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:39.458734 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd96d304-5234-4b7c-ad48-4ab9349430ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:39.460355 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-fd96d304-5234-4b7c-ad48-4ab9349430ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:39.576906 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd96d304-5234-4b7c-ad48-4ab9349430ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:49:39.578899 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd96d304-5234-4b7c-ad48-4ab9349430ed service neutron] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:49:39.605686 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 733/1461] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:49:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:49:41.580127 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de71a4d-aa45-4306-890e-9342abad5a8b None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:41.586582 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de71a4d-aa45-4306-890e-9342abad5a8b None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:41.586582 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de71a4d-aa45-4306-890e-9342abad5a8b None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:41.586582 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 729/1462] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 08:49:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:49:41.588145 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016fdd23-7f87-4603-a304-d34e33a72490 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:41.588562 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016fdd23-7f87-4603-a304-d34e33a72490 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:41.588716 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016fdd23-7f87-4603-a304-d34e33a72490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:41.655814 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-016fdd23-7f87-4603-a304-d34e33a72490 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:49:41.657080 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-016fdd23-7f87-4603-a304-d34e33a72490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:41.695439 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:41.754891 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 734/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 08:49:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:49:42.038609 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de71a4d-aa45-4306-890e-9342abad5a8b None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:49:42.040165 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-8de71a4d-aa45-4306-890e-9342abad5a8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:42.143795 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de71a4d-aa45-4306-890e-9342abad5a8b None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:49:42.181415 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:42.219854 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e3b18d-1b46-4191-b8d7-65c228a5d5ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:49:42.220480 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e3b18d-1b46-4191-b8d7-65c228a5d5ce service neutron] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:42.220800 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e3b18d-1b46-4191-b8d7-65c228a5d5ce service neutron] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:42.221051 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e3b18d-1b46-4191-b8d7-65c228a5d5ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:42.222642 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-f8e3b18d-1b46-4191-b8d7-65c228a5d5ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:42.342788 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e3b18d-1b46-4191-b8d7-65c228a5d5ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:49:42.344559 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e3b18d-1b46-4191-b8d7-65c228a5d5ce service neutron] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:49:42.364198 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 730/1464] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:49:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:49:44.342026 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431173d4-f42a-453d-bd26-b4fcaf4b46eb None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:44.342612 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431173d4-f42a-453d-bd26-b4fcaf4b46eb None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:44.342612 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431173d4-f42a-453d-bd26-b4fcaf4b46eb None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:44.343906 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 735/1465] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 08:49:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:49:44.349636 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efa24d9-e8bb-4ce0-bb23-0b48f2fcf6ae None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:44.350149 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efa24d9-e8bb-4ce0-bb23-0b48f2fcf6ae None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:44.350149 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efa24d9-e8bb-4ce0-bb23-0b48f2fcf6ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:44.417961 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-2efa24d9-e8bb-4ce0-bb23-0b48f2fcf6ae None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:49:44.419182 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-2efa24d9-e8bb-4ce0-bb23-0b48f2fcf6ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:44.437432 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:44.517797 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 731/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 08:49:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:49:44.801152 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431173d4-f42a-453d-bd26-b4fcaf4b46eb None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:49:44.802711 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-431173d4-f42a-453d-bd26-b4fcaf4b46eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:44.911027 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431173d4-f42a-453d-bd26-b4fcaf4b46eb None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:49:44.977505 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:44.984272 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9268c0-16bd-4791-9126-faf21776cd3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:49:44.990774 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9268c0-16bd-4791-9126-faf21776cd3d service neutron] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:44.990774 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9268c0-16bd-4791-9126-faf21776cd3d service neutron] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:44.990774 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9268c0-16bd-4791-9126-faf21776cd3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:44.990774 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-be9268c0-16bd-4791-9126-faf21776cd3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:45.096771 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9268c0-16bd-4791-9126-faf21776cd3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:49:45.098206 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9268c0-16bd-4791-9126-faf21776cd3d service neutron] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:49:45.116642 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 736/1467] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:49:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:49:47.089697 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df758688-69e4-4fb2-b52e-8a813a2f57da None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:47.090247 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df758688-69e4-4fb2-b52e-8a813a2f57da None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:47.090247 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df758688-69e4-4fb2-b52e-8a813a2f57da None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:47.091600 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 732/1468] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 08:49:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:49:47.097500 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee388c6-6b5c-471f-bfa3-80bffa142eed None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:47.097993 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee388c6-6b5c-471f-bfa3-80bffa142eed None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:47.098100 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee388c6-6b5c-471f-bfa3-80bffa142eed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:47.166457 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-6ee388c6-6b5c-471f-bfa3-80bffa142eed None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:49:47.167605 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-6ee388c6-6b5c-471f-bfa3-80bffa142eed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:47.217226 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:47.265718 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 737/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 08:49:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:49:47.550914 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df758688-69e4-4fb2-b52e-8a813a2f57da None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:49:47.552477 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-df758688-69e4-4fb2-b52e-8a813a2f57da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:47.656507 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df758688-69e4-4fb2-b52e-8a813a2f57da None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:49:47.703666 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:47.730756 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f8a57e-d324-45ab-bb7c-b9ac6be324d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:49:47.731358 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f8a57e-d324-45ab-bb7c-b9ac6be324d5 service neutron] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:47.731635 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f8a57e-d324-45ab-bb7c-b9ac6be324d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:47.731904 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f8a57e-d324-45ab-bb7c-b9ac6be324d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:47.733539 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-b0f8a57e-d324-45ab-bb7c-b9ac6be324d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:47.844631 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f8a57e-d324-45ab-bb7c-b9ac6be324d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:49:47.846073 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f8a57e-d324-45ab-bb7c-b9ac6be324d5 service neutron] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:49:47.866326 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 733/1470] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:49:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:49:57.274010 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=22 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:57.864004 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=66 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:50:07.581011 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c83ab3-1fca-4258-8b00-3585efc42f73 None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:50:07.581573 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c83ab3-1fca-4258-8b00-3585efc42f73 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:50:07.581573 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c83ab3-1fca-4258-8b00-3585efc42f73 None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:50:07.582930 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 738/1471] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:50:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:50:07.589114 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e54166b-dbe1-451d-846b-47665000d0ec None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:50:07.590033 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e54166b-dbe1-451d-846b-47665000d0ec None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:50:07.590033 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e54166b-dbe1-451d-846b-47665000d0ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:50:07.657075 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-6e54166b-dbe1-451d-846b-47665000d0ec None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:50:07.658322 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-6e54166b-dbe1-451d-846b-47665000d0ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:50:07.758659 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 734/1472] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:50:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:50:08.054465 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a72ffd4-d8a5-4070-bb2b-00d0a85e07e4 None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:50:08.054465 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a72ffd4-d8a5-4070-bb2b-00d0a85e07e4 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:50:08.054465 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a72ffd4-d8a5-4070-bb2b-00d0a85e07e4 None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:50:08.055780 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 739/1473] 10.222.0.55 () {54 vars in 967 bytes} [Wed Aug 24 08:50:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:50:08.060837 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f33329e-05b6-492a-ae94-45d6f6ab56c7 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:50:08.061129 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f33329e-05b6-492a-ae94-45d6f6ab56c7 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:50:08.061508 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f33329e-05b6-492a-ae94-45d6f6ab56c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:50:08.140259 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-9f33329e-05b6-492a-ae94-45d6f6ab56c7 None None] MFA Rules not processed for user `5512043ca9a7421b821741ca88d50b07`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:50:08.141510 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-9f33329e-05b6-492a-ae94-45d6f6ab56c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:50:08.162883 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:50:08.222659 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 735/1474] 10.222.0.55 () {58 vars in 1064 bytes} [Wed Aug 24 08:50:08 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:50:08.230555 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a72ffd4-d8a5-4070-bb2b-00d0a85e07e4 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:50:08.232647 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-0a72ffd4-d8a5-4070-bb2b-00d0a85e07e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:50:08.345315 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a72ffd4-d8a5-4070-bb2b-00d0a85e07e4 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:50:08.433666 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6418afcb-6b05-4f0c-af70-3086875a192b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5512043ca9a7421b821741ca88d50b07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:50:08.434626 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6418afcb-6b05-4f0c-af70-3086875a192b service glance] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:50:08.435062 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6418afcb-6b05-4f0c-af70-3086875a192b service glance] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:50:08.435500 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6418afcb-6b05-4f0c-af70-3086875a192b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:50:08.438495 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-6418afcb-6b05-4f0c-af70-3086875a192b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:50:08.495969 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:50:08.553732 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6418afcb-6b05-4f0c-af70-3086875a192b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:50:08.555865 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6418afcb-6b05-4f0c-af70-3086875a192b service glance] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:50:08.575699 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 740/1475] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:50:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:50:10.616437 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d7cc6f-fbd2-4bfd-ac3f-f2122bfa83b9 None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:50:10.617008 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d7cc6f-fbd2-4bfd-ac3f-f2122bfa83b9 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:50:10.617008 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d7cc6f-fbd2-4bfd-ac3f-f2122bfa83b9 None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:50:10.618369 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 736/1476] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:50:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:50:10.625009 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d4b1dc-123c-40db-8bf7-4bb7b800dee6 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:50:10.625563 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d4b1dc-123c-40db-8bf7-4bb7b800dee6 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:50:10.625704 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d4b1dc-123c-40db-8bf7-4bb7b800dee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:50:10.702218 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-c8d4b1dc-123c-40db-8bf7-4bb7b800dee6 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:50:10.703765 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-c8d4b1dc-123c-40db-8bf7-4bb7b800dee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:50:10.805053 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 741/1477] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:50:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:50:11.076593 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d7cc6f-fbd2-4bfd-ac3f-f2122bfa83b9 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:50:11.078122 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-a1d7cc6f-fbd2-4bfd-ac3f-f2122bfa83b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:50:11.208759 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d7cc6f-fbd2-4bfd-ac3f-f2122bfa83b9 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:50:11.259757 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:50:11.308222 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f11445-7455-4a1b-9822-82571038e0fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:50:11.308866 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f11445-7455-4a1b-9822-82571038e0fd admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:50:11.309144 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f11445-7455-4a1b-9822-82571038e0fd admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:50:11.309411 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f11445-7455-4a1b-9822-82571038e0fd admin admin] PATH_INFO: `/v3/projects` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:50:11.310048 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f11445-7455-4a1b-9822-82571038e0fd admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:50:11.311817 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: WARNING py.warnings [None req-39f11445-7455-4a1b-9822-82571038e0fd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 08:50:11.311817 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: warnings.warn(msg) Aug 24 08:50:11.311817 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: Aug 24 08:50:11.312993 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f11445-7455-4a1b-9822-82571038e0fd admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:50:11.333126 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 737/1478] 10.222.0.55 () {56 vars in 1108 bytes} [Wed Aug 24 08:50:11 2022] GET /identity/v3/projects => generated 2553 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:50:13.282490 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94d1cc0-876d-464c-a0e1-2cae61a75bfe None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:50:13.283183 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94d1cc0-876d-464c-a0e1-2cae61a75bfe None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:50:13.283183 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94d1cc0-876d-464c-a0e1-2cae61a75bfe None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:50:13.284356 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 742/1479] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:50:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:50:13.290531 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c4c64e-0f83-401d-98d0-67913da87c40 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:50:13.291025 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c4c64e-0f83-401d-98d0-67913da87c40 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:50:13.291139 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c4c64e-0f83-401d-98d0-67913da87c40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:50:13.357726 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-c9c4c64e-0f83-401d-98d0-67913da87c40 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:50:13.358986 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-c9c4c64e-0f83-401d-98d0-67913da87c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:50:13.457699 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 738/1480] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:50:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:50:13.751360 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94d1cc0-876d-464c-a0e1-2cae61a75bfe None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:50:13.753113 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-d94d1cc0-876d-464c-a0e1-2cae61a75bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:50:13.763518 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:50:13.901213 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94d1cc0-876d-464c-a0e1-2cae61a75bfe None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:50:13.983539 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:50:14.016145 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdf1ae4-6d84-403c-8c3b-8b602a2dd3d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17da357e555420799ee681bffbcc268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:50:14.016693 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdf1ae4-6d84-403c-8c3b-8b602a2dd3d3 service nova] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:50:14.016970 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdf1ae4-6d84-403c-8c3b-8b602a2dd3d3 service nova] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:50:14.017240 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdf1ae4-6d84-403c-8c3b-8b602a2dd3d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:50:14.018849 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-6cdf1ae4-6d84-403c-8c3b-8b602a2dd3d3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:50:14.110827 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdf1ae4-6d84-403c-8c3b-8b602a2dd3d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:50:14.112426 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdf1ae4-6d84-403c-8c3b-8b602a2dd3d3 service nova] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:50:14.130883 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 743/1481] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:50:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:50:16.247759 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb56fb7-73ef-4262-abcf-221a01372ea4 None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:50:16.248327 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb56fb7-73ef-4262-abcf-221a01372ea4 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:50:16.248327 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb56fb7-73ef-4262-abcf-221a01372ea4 None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:50:16.249705 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 739/1482] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:50:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:50:16.255064 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8011f928-9a8b-40da-9b3a-0275240a2a6a None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:50:16.255505 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8011f928-9a8b-40da-9b3a-0275240a2a6a None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:50:16.255630 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8011f928-9a8b-40da-9b3a-0275240a2a6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:50:16.342451 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-8011f928-9a8b-40da-9b3a-0275240a2a6a None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:50:16.343676 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-8011f928-9a8b-40da-9b3a-0275240a2a6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:50:16.391606 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:50:16.442868 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 744/1483] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:50:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:50:16.729745 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb56fb7-73ef-4262-abcf-221a01372ea4 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:50:16.731226 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-ccb56fb7-73ef-4262-abcf-221a01372ea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:50:16.754856 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:50:16.839510 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb56fb7-73ef-4262-abcf-221a01372ea4 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:50:16.921919 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac86efd-46e5-4a4c-95c3-20781f4cb987 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:50:16.922526 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac86efd-46e5-4a4c-95c3-20781f4cb987 service neutron] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:50:16.922806 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac86efd-46e5-4a4c-95c3-20781f4cb987 service neutron] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:50:16.923080 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac86efd-46e5-4a4c-95c3-20781f4cb987 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:50:16.924818 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-dac86efd-46e5-4a4c-95c3-20781f4cb987 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:50:17.002949 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:50:17.040344 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac86efd-46e5-4a4c-95c3-20781f4cb987 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:50:17.041825 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac86efd-46e5-4a4c-95c3-20781f4cb987 service neutron] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:50:17.060883 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 740/1484] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:50:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:50:19.067629 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7e5136-7a83-414b-9971-320a7a1f1a2a None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:50:19.068192 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7e5136-7a83-414b-9971-320a7a1f1a2a None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:50:19.068192 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7e5136-7a83-414b-9971-320a7a1f1a2a None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:50:19.070165 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 745/1485] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:50:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:50:19.076039 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388abc69-c77e-4509-9bf3-016952094f4a None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:50:19.076598 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388abc69-c77e-4509-9bf3-016952094f4a None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:50:19.076715 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388abc69-c77e-4509-9bf3-016952094f4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:50:19.144405 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-388abc69-c77e-4509-9bf3-016952094f4a None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:50:19.145916 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-388abc69-c77e-4509-9bf3-016952094f4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:50:19.244130 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 741/1486] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:50:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:50:19.538605 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7e5136-7a83-414b-9971-320a7a1f1a2a None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:50:19.540187 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-8f7e5136-7a83-414b-9971-320a7a1f1a2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:50:19.648303 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7e5136-7a83-414b-9971-320a7a1f1a2a None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:50:19.723466 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:50:19.725331 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbb7b32-53ff-4544-84b0-050da6bbdb8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:50:19.726524 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbb7b32-53ff-4544-84b0-050da6bbdb8e service neutron] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:50:19.726803 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbb7b32-53ff-4544-84b0-050da6bbdb8e service neutron] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:50:19.727089 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbb7b32-53ff-4544-84b0-050da6bbdb8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:50:19.728773 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-9bbb7b32-53ff-4544-84b0-050da6bbdb8e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:50:19.842613 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbb7b32-53ff-4544-84b0-050da6bbdb8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:50:19.844044 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbb7b32-53ff-4544-84b0-050da6bbdb8e service neutron] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:50:19.863086 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 746/1487] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:50:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:50:21.873979 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03c415f-513d-468e-a9d2-1c82293ddf4b None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:50:21.874563 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03c415f-513d-468e-a9d2-1c82293ddf4b None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:50:21.874563 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03c415f-513d-468e-a9d2-1c82293ddf4b None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:50:21.875857 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 742/1488] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:50:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:50:21.881338 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9bbfd6-4075-45c8-aea4-5a6c3a671e74 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:50:21.881872 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9bbfd6-4075-45c8-aea4-5a6c3a671e74 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:50:21.881872 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9bbfd6-4075-45c8-aea4-5a6c3a671e74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:50:21.952358 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-2e9bbfd6-4075-45c8-aea4-5a6c3a671e74 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:50:21.953624 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-2e9bbfd6-4075-45c8-aea4-5a6c3a671e74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:50:22.008829 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:50:22.057923 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 747/1489] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:50:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:50:22.344532 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03c415f-513d-468e-a9d2-1c82293ddf4b None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:50:22.346088 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-b03c415f-513d-468e-a9d2-1c82293ddf4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:50:22.382996 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:50:22.454286 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03c415f-513d-468e-a9d2-1c82293ddf4b None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:50:22.531924 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19547c2-8971-4252-ba65-b305e6c019c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:50:22.532568 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19547c2-8971-4252-ba65-b305e6c019c7 service neutron] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:50:22.532845 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19547c2-8971-4252-ba65-b305e6c019c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:50:22.533121 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19547c2-8971-4252-ba65-b305e6c019c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:50:22.534764 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-b19547c2-8971-4252-ba65-b305e6c019c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:50:22.617266 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:50:22.649753 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19547c2-8971-4252-ba65-b305e6c019c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:50:22.651554 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19547c2-8971-4252-ba65-b305e6c019c7 service neutron] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:50:22.672243 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 743/1490] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:50:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:50:25.010815 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707c65bd-1b19-41d4-b359-6010beae65dc None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:50:25.011389 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707c65bd-1b19-41d4-b359-6010beae65dc None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:50:25.011389 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707c65bd-1b19-41d4-b359-6010beae65dc None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:50:25.012750 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 748/1491] 10.222.0.55 () {54 vars in 927 bytes} [Wed Aug 24 08:50:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:50:25.019315 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03601496-d1fd-409e-9dd6-8c80524fe72f None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:50:25.019804 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03601496-d1fd-409e-9dd6-8c80524fe72f None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:50:25.019915 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03601496-d1fd-409e-9dd6-8c80524fe72f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:50:25.092273 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-03601496-d1fd-409e-9dd6-8c80524fe72f None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:50:25.093525 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-03601496-d1fd-409e-9dd6-8c80524fe72f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:50:25.192715 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 744/1492] 10.222.0.55 () {58 vars in 1024 bytes} [Wed Aug 24 08:50:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:50:25.482269 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707c65bd-1b19-41d4-b359-6010beae65dc None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:50:25.483812 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-707c65bd-1b19-41d4-b359-6010beae65dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:50:25.593399 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707c65bd-1b19-41d4-b359-6010beae65dc None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:50:25.669495 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9bb1c9-439e-4a23-a9eb-a5327582072d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:50:25.670113 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9bb1c9-439e-4a23-a9eb-a5327582072d service neutron] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:50:25.670390 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9bb1c9-439e-4a23-a9eb-a5327582072d service neutron] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:50:25.670662 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9bb1c9-439e-4a23-a9eb-a5327582072d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:50:25.672317 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-1a9bb1c9-439e-4a23-a9eb-a5327582072d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:50:25.682092 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:50:25.789452 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9bb1c9-439e-4a23-a9eb-a5327582072d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:50:25.790771 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9bb1c9-439e-4a23-a9eb-a5327582072d service neutron] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:50:25.809648 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 749/1493] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:50:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:50:35.190738 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=82 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:50:35.807203 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=50 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:51:48.341160 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5170bf-84c7-40b2-a24a-34521fc10c11 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:51:48.341754 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5170bf-84c7-40b2-a24a-34521fc10c11 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:51:48.341754 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5170bf-84c7-40b2-a24a-34521fc10c11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:51:48.419062 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-af5170bf-84c7-40b2-a24a-34521fc10c11 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:51:48.420202 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-af5170bf-84c7-40b2-a24a-34521fc10c11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:51:48.523263 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 745/1494] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:51:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:51:48.531718 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922c502f-4c3a-4846-abf5-97e6dd54a4d2 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:51:48.532321 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922c502f-4c3a-4846-abf5-97e6dd54a4d2 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:51:48.532321 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922c502f-4c3a-4846-abf5-97e6dd54a4d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:51:48.610340 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-922c502f-4c3a-4846-abf5-97e6dd54a4d2 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:51:48.611563 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-922c502f-4c3a-4846-abf5-97e6dd54a4d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:51:48.712077 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 750/1495] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:51:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:51:48.716831 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5170bf-84c7-40b2-a24a-34521fc10c11 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:51:48.727447 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-af5170bf-84c7-40b2-a24a-34521fc10c11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:51:48.730137 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5170bf-84c7-40b2-a24a-34521fc10c11 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:51:48.792178 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:51:48.825464 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d115c6c-6caf-4206-b077-d80924ee4c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:51:48.826065 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d115c6c-6caf-4206-b077-d80924ee4c71 admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:51:48.826337 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d115c6c-6caf-4206-b077-d80924ee4c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:51:48.826623 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d115c6c-6caf-4206-b077-d80924ee4c71 admin admin] PATH_INFO: `/v3/domains` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:51:48.827256 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d115c6c-6caf-4206-b077-d80924ee4c71 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:51:48.827547 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d115c6c-6caf-4206-b077-d80924ee4c71 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:51:48.828647 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: WARNING py.warnings [None req-0d115c6c-6caf-4206-b077-d80924ee4c71 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 08:51:48.828647 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: warnings.warn(msg) Aug 24 08:51:48.828647 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: Aug 24 08:51:48.829528 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d115c6c-6caf-4206-b077-d80924ee4c71 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:51:48.841254 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 746/1496] 10.222.0.55 () {58 vars in 1153 bytes} [Wed Aug 24 08:51:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:51:48.850458 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922c502f-4c3a-4846-abf5-97e6dd54a4d2 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:51:48.859775 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922c502f-4c3a-4846-abf5-97e6dd54a4d2 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:51:48.921679 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:51:48.955084 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55690369-3ee1-46c3-9627-a7acd0552ede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:51:48.955682 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55690369-3ee1-46c3-9627-a7acd0552ede admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:51:48.955958 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55690369-3ee1-46c3-9627-a7acd0552ede admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:51:48.956226 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55690369-3ee1-46c3-9627-a7acd0552ede admin admin] PATH_INFO: `/v3/projects` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:51:48.956864 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55690369-3ee1-46c3-9627-a7acd0552ede admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:51:48.959146 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: WARNING py.warnings [None req-55690369-3ee1-46c3-9627-a7acd0552ede admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 08:51:48.959146 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: warnings.warn(msg) Aug 24 08:51:48.959146 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: Aug 24 08:51:48.960338 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55690369-3ee1-46c3-9627-a7acd0552ede admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:51:49.103475 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 751/1497] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 08:51:48 2022] POST /identity/v3/projects => generated 379 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:51:49.107723 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d115c6c-6caf-4206-b077-d80924ee4c71 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:51:49.109286 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-0d115c6c-6caf-4206-b077-d80924ee4c71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:51:49.144838 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d115c6c-6caf-4206-b077-d80924ee4c71 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:51:49.238040 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faeb466-2bc6-420f-a2ac-2b117d5cb572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:51:49.239134 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faeb466-2bc6-420f-a2ac-2b117d5cb572 admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:51:49.239410 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faeb466-2bc6-420f-a2ac-2b117d5cb572 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:51:49.239722 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faeb466-2bc6-420f-a2ac-2b117d5cb572 admin admin] PATH_INFO: `/v3/users` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:51:49.240353 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faeb466-2bc6-420f-a2ac-2b117d5cb572 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:51:49.241492 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: WARNING py.warnings [None req-0faeb466-2bc6-420f-a2ac-2b117d5cb572 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 08:51:49.241492 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: warnings.warn(msg) Aug 24 08:51:49.241492 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: Aug 24 08:51:49.242704 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faeb466-2bc6-420f-a2ac-2b117d5cb572 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:51:49.340970 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 747/1498] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 08:51:49 2022] POST /identity/v3/users => generated 290 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:51:49.345129 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55690369-3ee1-46c3-9627-a7acd0552ede admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:51:49.350933 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-55690369-3ee1-46c3-9627-a7acd0552ede admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:51:49.409291 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55690369-3ee1-46c3-9627-a7acd0552ede admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:51:49.498488 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:51:49.508836 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209fdcc3-4dc4-4b74-b4f7-2ceab8c08e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:51:49.509448 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209fdcc3-4dc4-4b74-b4f7-2ceab8c08e93 admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:51:49.509723 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209fdcc3-4dc4-4b74-b4f7-2ceab8c08e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:51:49.509990 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209fdcc3-4dc4-4b74-b4f7-2ceab8c08e93 admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:51:49.510624 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209fdcc3-4dc4-4b74-b4f7-2ceab8c08e93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:51:49.511637 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: WARNING py.warnings [None req-209fdcc3-4dc4-4b74-b4f7-2ceab8c08e93 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 08:51:49.511637 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: warnings.warn(msg) Aug 24 08:51:49.511637 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: Aug 24 08:51:49.512553 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209fdcc3-4dc4-4b74-b4f7-2ceab8c08e93 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:51:49.525472 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 752/1499] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:51:49 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:51:49.530386 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faeb466-2bc6-420f-a2ac-2b117d5cb572 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:51:49.531137 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-0faeb466-2bc6-420f-a2ac-2b117d5cb572 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:51:49.543180 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faeb466-2bc6-420f-a2ac-2b117d5cb572 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:51:49.569750 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:51:49.648893 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee88709e-451d-4ff9-9d2f-6b81d75525ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:51:49.649530 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee88709e-451d-4ff9-9d2f-6b81d75525ce admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:51:49.649852 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee88709e-451d-4ff9-9d2f-6b81d75525ce admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:51:49.650068 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee88709e-451d-4ff9-9d2f-6b81d75525ce admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:51:49.650701 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee88709e-451d-4ff9-9d2f-6b81d75525ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:51:49.651732 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: WARNING py.warnings [None req-ee88709e-451d-4ff9-9d2f-6b81d75525ce admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 08:51:49.651732 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: warnings.warn(msg) Aug 24 08:51:49.651732 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: Aug 24 08:51:49.652667 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee88709e-451d-4ff9-9d2f-6b81d75525ce admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:51:49.666339 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 748/1500] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:51:49 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:51:49.670566 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209fdcc3-4dc4-4b74-b4f7-2ceab8c08e93 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:51:49.672165 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-209fdcc3-4dc4-4b74-b4f7-2ceab8c08e93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:51:49.708487 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209fdcc3-4dc4-4b74-b4f7-2ceab8c08e93 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:51:49.923062 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347a1e6c-8f79-4a57-9eed-04297cf56754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:51:49.923657 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347a1e6c-8f79-4a57-9eed-04297cf56754 admin admin] REQUEST_METHOD: `PUT` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:51:49.923936 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347a1e6c-8f79-4a57-9eed-04297cf56754 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:51:49.924215 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347a1e6c-8f79-4a57-9eed-04297cf56754 admin admin] PATH_INFO: `/v3/projects/12b83acd34464c27ba1bc8e9e7518cfb/users/b418c089f30f43909b2ffe698f743c20/roles/b674fd0a592e4e0cada811bba5ceea4d` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:51:49.947002 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347a1e6c-8f79-4a57-9eed-04297cf56754 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12b83acd34464c27ba1bc8e9e7518cfb, user_id=b418c089f30f43909b2ffe698f743c20, role_id=b674fd0a592e4e0cada811bba5ceea4d)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:51:49.948039 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: WARNING py.warnings [None req-347a1e6c-8f79-4a57-9eed-04297cf56754 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 08:51:49.948039 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: warnings.warn(msg) Aug 24 08:51:49.948039 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: Aug 24 08:51:49.950006 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347a1e6c-8f79-4a57-9eed-04297cf56754 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:51:50.074592 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 753/1501] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 08:51:49 2022] PUT /identity/v3/projects/12b83acd34464c27ba1bc8e9e7518cfb/users/b418c089f30f43909b2ffe698f743c20/roles/b674fd0a592e4e0cada811bba5ceea4d => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:51:50.086585 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991a45e9-9cfa-45c3-bbb9-01674d315b8c None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:51:50.086919 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991a45e9-9cfa-45c3-bbb9-01674d315b8c None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:51:50.087193 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991a45e9-9cfa-45c3-bbb9-01674d315b8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:51:50.147079 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-991a45e9-9cfa-45c3-bbb9-01674d315b8c None None] MFA Rules not processed for user `b418c089f30f43909b2ffe698f743c20`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:51:50.148303 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-991a45e9-9cfa-45c3-bbb9-01674d315b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:51:50.227651 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 749/1502] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:51:50 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:51:50.241755 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256765a6-c8de-4445-abf4-50121f9b32fe None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:51:50.242043 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256765a6-c8de-4445-abf4-50121f9b32fe None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:51:50.242471 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256765a6-c8de-4445-abf4-50121f9b32fe None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:51:50.243841 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 754/1503] 10.222.0.55 () {54 vars in 964 bytes} [Wed Aug 24 08:51:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:51:50.248856 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c466eb29-ba31-4d66-9d26-bbfe8df9f3c3 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:51:50.253094 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c466eb29-ba31-4d66-9d26-bbfe8df9f3c3 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:51:50.253094 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c466eb29-ba31-4d66-9d26-bbfe8df9f3c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:51:50.280698 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:51:50.329326 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-c466eb29-ba31-4d66-9d26-bbfe8df9f3c3 None None] MFA Rules not processed for user `fbbb9ac667f04c35a63b2689df18efed`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:51:50.330569 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-c466eb29-ba31-4d66-9d26-bbfe8df9f3c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:51:50.408117 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 750/1504] 10.222.0.55 () {58 vars in 1061 bytes} [Wed Aug 24 08:51:50 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:51:50.416785 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256765a6-c8de-4445-abf4-50121f9b32fe None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:51:50.418302 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-256765a6-c8de-4445-abf4-50121f9b32fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:51:50.470478 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:51:50.529415 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256765a6-c8de-4445-abf4-50121f9b32fe None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:51:50.608552 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c08d692-1b97-4b7e-bcfe-f8bc799c0ea0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbb9ac667f04c35a63b2689df18efed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:51:50.609234 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c08d692-1b97-4b7e-bcfe-f8bc799c0ea0 service cinder] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:51:50.609510 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c08d692-1b97-4b7e-bcfe-f8bc799c0ea0 service cinder] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:51:50.609781 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c08d692-1b97-4b7e-bcfe-f8bc799c0ea0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:51:50.611385 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-3c08d692-1b97-4b7e-bcfe-f8bc799c0ea0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:51:50.718163 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c08d692-1b97-4b7e-bcfe-f8bc799c0ea0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:51:50.719606 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c08d692-1b97-4b7e-bcfe-f8bc799c0ea0 service cinder] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:51:50.723224 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:51:50.739913 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 755/1505] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:51:50 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:51:50.785936 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d599718-d750-4881-82a8-439ffa2d84fd None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:51:50.786590 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d599718-d750-4881-82a8-439ffa2d84fd None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:51:50.786590 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d599718-d750-4881-82a8-439ffa2d84fd None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:51:50.787737 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 751/1506] 10.222.0.55 () {52 vars in 830 bytes} [Wed Aug 24 08:51:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:51:50.791535 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c08d692-1b97-4b7e-bcfe-f8bc799c0ea0 service cinder] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:51:50.792808 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-3c08d692-1b97-4b7e-bcfe-f8bc799c0ea0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:51:50.895689 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c08d692-1b97-4b7e-bcfe-f8bc799c0ea0 service cinder] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:51:50.994262 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e44fa31-1d51-4ebd-8841-9a7d4d40af6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:51:50.994808 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e44fa31-1d51-4ebd-8841-9a7d4d40af6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:51:50.995088 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e44fa31-1d51-4ebd-8841-9a7d4d40af6c admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:51:50.995356 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e44fa31-1d51-4ebd-8841-9a7d4d40af6c admin admin] PATH_INFO: `/v3/users/b418c089f30f43909b2ffe698f743c20` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:51:51.007894 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e44fa31-1d51-4ebd-8841-9a7d4d40af6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b418c089f30f43909b2ffe698f743c20)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:51:51.008939 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: WARNING py.warnings [None req-0e44fa31-1d51-4ebd-8841-9a7d4d40af6c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 08:51:51.008939 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: warnings.warn(msg) Aug 24 08:51:51.008939 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: Aug 24 08:51:51.010154 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e44fa31-1d51-4ebd-8841-9a7d4d40af6c admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:51:51.020269 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:51:51.299206 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-0e44fa31-1d51-4ebd-8841-9a7d4d40af6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b418c089f30f43909b2ffe698f743c20'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:51:51.299551 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-0e44fa31-1d51-4ebd-8841-9a7d4d40af6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b418c089f30f43909b2ffe698f743c20'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:51:51.300067 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-0e44fa31-1d51-4ebd-8841-9a7d4d40af6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b418c089f30f43909b2ffe698f743c20'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:51:51.324257 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-0e44fa31-1d51-4ebd-8841-9a7d4d40af6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b418c089f30f43909b2ffe698f743c20'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:51:51.437972 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-0e44fa31-1d51-4ebd-8841-9a7d4d40af6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b418c089f30f43909b2ffe698f743c20'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:51:51.458471 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 756/1507] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 08:51:50 2022] DELETE /identity/v3/users/b418c089f30f43909b2ffe698f743c20 => generated 0 bytes in 667 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:51:51.464967 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5350264-aca3-4659-9dc0-3ce19b6a5059 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:51:51.464967 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5350264-aca3-4659-9dc0-3ce19b6a5059 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:51:51.464967 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5350264-aca3-4659-9dc0-3ce19b6a5059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:51:51.522177 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-c5350264-aca3-4659-9dc0-3ce19b6a5059 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:51:51.523396 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-c5350264-aca3-4659-9dc0-3ce19b6a5059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:51:51.598926 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:51:51.611657 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 752/1508] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:51:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:51:51.621466 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e44fa31-1d51-4ebd-8841-9a7d4d40af6c admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:51:51.622700 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-0e44fa31-1d51-4ebd-8841-9a7d4d40af6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:51:51.727361 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e44fa31-1d51-4ebd-8841-9a7d4d40af6c admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:51:51.784328 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:51:51.804266 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdd9b0c-07eb-4738-a712-8d54aa8081fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:51:51.804904 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdd9b0c-07eb-4738-a712-8d54aa8081fa service neutron] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:51:51.805180 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdd9b0c-07eb-4738-a712-8d54aa8081fa service neutron] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:51:51.805447 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdd9b0c-07eb-4738-a712-8d54aa8081fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:51:51.807045 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-1fdd9b0c-07eb-4738-a712-8d54aa8081fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:51:51.914328 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdd9b0c-07eb-4738-a712-8d54aa8081fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:51:51.915683 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdd9b0c-07eb-4738-a712-8d54aa8081fa service neutron] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:51:51.934749 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 757/1509] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:51:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:51:52.418549 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5350264-aca3-4659-9dc0-3ce19b6a5059 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:51:52.419856 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-c5350264-aca3-4659-9dc0-3ce19b6a5059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:51:52.431465 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5350264-aca3-4659-9dc0-3ce19b6a5059 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:51:52.535567 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01462b9-8d30-43fc-b841-650458f747de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:51:52.536220 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01462b9-8d30-43fc-b841-650458f747de admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:51:52.536472 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01462b9-8d30-43fc-b841-650458f747de admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:51:52.536870 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01462b9-8d30-43fc-b841-650458f747de admin admin] PATH_INFO: `/v3/projects/12b83acd34464c27ba1bc8e9e7518cfb` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:51:52.547552 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01462b9-8d30-43fc-b841-650458f747de admin admin] RBAC: Authorizing `identity:delete_project(project_id=12b83acd34464c27ba1bc8e9e7518cfb)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:51:52.548592 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: WARNING py.warnings [None req-e01462b9-8d30-43fc-b841-650458f747de admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 08:51:52.548592 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: warnings.warn(msg) Aug 24 08:51:52.548592 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: Aug 24 08:51:52.549785 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01462b9-8d30-43fc-b841-650458f747de admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:51:52.667568 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-e01462b9-8d30-43fc-b841-650458f747de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12b83acd34464c27ba1bc8e9e7518cfb', 'request_id': 'req-e01462b9-8d30-43fc-b841-650458f747de'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:51:52.736391 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-e01462b9-8d30-43fc-b841-650458f747de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12b83acd34464c27ba1bc8e9e7518cfb', 'request_id': 'req-e01462b9-8d30-43fc-b841-650458f747de'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:51:52.750995 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-e01462b9-8d30-43fc-b841-650458f747de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12b83acd34464c27ba1bc8e9e7518cfb', 'request_id': 'req-e01462b9-8d30-43fc-b841-650458f747de'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:51:52.775849 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-e01462b9-8d30-43fc-b841-650458f747de admin admin] The token cache is being invalidate because project 12b83acd34464c27ba1bc8e9e7518cfb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:51:52.776158 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-e01462b9-8d30-43fc-b841-650458f747de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:51:52.776677 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-e01462b9-8d30-43fc-b841-650458f747de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:51:52.777953 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 753/1510] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 08:51:52 2022] DELETE /identity/v3/projects/12b83acd34464c27ba1bc8e9e7518cfb => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:52:01.932517 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=62 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:52:02.773945 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:53:27.364284 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2598530c-0815-4349-9c4c-394ace7fd00e None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:53:27.364942 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2598530c-0815-4349-9c4c-394ace7fd00e None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:53:27.364942 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2598530c-0815-4349-9c4c-394ace7fd00e None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:53:27.366185 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 758/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 08:53:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:53:27.371536 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19371b2-29dd-4ea8-9cce-3d03a07538d9 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:53:27.372139 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19371b2-29dd-4ea8-9cce-3d03a07538d9 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:53:27.372139 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19371b2-29dd-4ea8-9cce-3d03a07538d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:53:27.452334 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-d19371b2-29dd-4ea8-9cce-3d03a07538d9 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:53:27.453515 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-d19371b2-29dd-4ea8-9cce-3d03a07538d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:53:27.563509 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 754/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 08:53:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:53:27.716219 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2598530c-0815-4349-9c4c-394ace7fd00e None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:53:27.720136 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-2598530c-0815-4349-9c4c-394ace7fd00e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:53:27.841554 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2598530c-0815-4349-9c4c-394ace7fd00e None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:53:27.847392 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd450eab-3e94-462f-a20a-2d8fce5364b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17da357e555420799ee681bffbcc268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:53:27.847997 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd450eab-3e94-462f-a20a-2d8fce5364b1 service nova] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:53:27.848276 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd450eab-3e94-462f-a20a-2d8fce5364b1 service nova] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:53:27.848565 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd450eab-3e94-462f-a20a-2d8fce5364b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:53:27.856998 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd450eab-3e94-462f-a20a-2d8fce5364b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:53:27.858571 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd450eab-3e94-462f-a20a-2d8fce5364b1 service nova] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:53:27.867756 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 759/1513] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:53:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:53:28.692178 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c769fd2f-6459-484c-a917-096f8bfa59f5 None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:53:28.693119 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c769fd2f-6459-484c-a917-096f8bfa59f5 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:53:28.693119 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c769fd2f-6459-484c-a917-096f8bfa59f5 None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:53:28.694686 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 755/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 08:53:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:53:28.700069 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04164dcc-6d07-4ccc-a102-a7bb1b8bbdb3 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:53:28.700508 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04164dcc-6d07-4ccc-a102-a7bb1b8bbdb3 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:53:28.700649 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04164dcc-6d07-4ccc-a102-a7bb1b8bbdb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:53:28.719808 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-04164dcc-6d07-4ccc-a102-a7bb1b8bbdb3 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:53:28.721044 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-04164dcc-6d07-4ccc-a102-a7bb1b8bbdb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:53:28.728841 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 760/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 08:53:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:53:28.886435 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c769fd2f-6459-484c-a917-096f8bfa59f5 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:53:28.900822 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c769fd2f-6459-484c-a917-096f8bfa59f5 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:53:28.906219 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723300dd-bc17-41f8-950d-83ee46d25b00 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17da357e555420799ee681bffbcc268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:53:28.906804 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723300dd-bc17-41f8-950d-83ee46d25b00 service nova] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:53:28.907082 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723300dd-bc17-41f8-950d-83ee46d25b00 service nova] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:53:28.907349 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723300dd-bc17-41f8-950d-83ee46d25b00 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:53:28.915960 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723300dd-bc17-41f8-950d-83ee46d25b00 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:53:28.917566 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723300dd-bc17-41f8-950d-83ee46d25b00 service nova] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:53:28.926940 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 756/1516] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:53:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:53:29.294865 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad87e528-02aa-4802-a084-2ff82a6d12d1 None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:53:29.295078 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad87e528-02aa-4802-a084-2ff82a6d12d1 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:53:29.295323 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad87e528-02aa-4802-a084-2ff82a6d12d1 None None] PATH_INFO: `/v3` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:53:29.296619 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 761/1517] 10.222.0.59 () {54 vars in 925 bytes} [Wed Aug 24 08:53:29 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:53:29.303139 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a1ff7f-f390-405a-bc43-646e58ed2c1a None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:53:29.303418 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a1ff7f-f390-405a-bc43-646e58ed2c1a None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:53:29.303694 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a1ff7f-f390-405a-bc43-646e58ed2c1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:53:29.363677 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-93a1ff7f-f390-405a-bc43-646e58ed2c1a None None] MFA Rules not processed for user `cf0c99771661413aaa0f237af6b394ab`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:53:29.364957 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-93a1ff7f-f390-405a-bc43-646e58ed2c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:53:29.381504 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:53:29.435446 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 757/1518] 10.222.0.59 () {58 vars in 1015 bytes} [Wed Aug 24 08:53:29 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:53:38.715038 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=56 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:53:38.905451 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8f323a-26dc-4640-be53-21bc5c9a058f None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:53:38.905652 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8f323a-26dc-4640-be53-21bc5c9a058f None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:53:38.905901 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8f323a-26dc-4640-be53-21bc5c9a058f None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:53:38.907237 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 762/1519] 10.222.0.55 () {54 vars in 961 bytes} [Wed Aug 24 08:53:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:53:38.912289 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fb3faa-0555-4ad0-accd-cfed770efbf1 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:53:38.912776 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fb3faa-0555-4ad0-accd-cfed770efbf1 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:53:38.912890 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fb3faa-0555-4ad0-accd-cfed770efbf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:53:38.932331 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-92fb3faa-0555-4ad0-accd-cfed770efbf1 None None] MFA Rules not processed for user `cf0c99771661413aaa0f237af6b394ab`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:53:38.933558 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-92fb3faa-0555-4ad0-accd-cfed770efbf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:53:38.942050 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 758/1520] 10.222.0.55 () {58 vars in 1058 bytes} [Wed Aug 24 08:53:38 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:53:38.950059 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8f323a-26dc-4640-be53-21bc5c9a058f None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:53:38.963968 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8f323a-26dc-4640-be53-21bc5c9a058f None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:53:38.968989 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7598895f-d50c-4346-ab68-45e7b9db5ffd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf0c99771661413aaa0f237af6b394ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:53:38.969577 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7598895f-d50c-4346-ab68-45e7b9db5ffd service placement] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:53:38.969859 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7598895f-d50c-4346-ab68-45e7b9db5ffd service placement] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:53:38.970129 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7598895f-d50c-4346-ab68-45e7b9db5ffd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:53:38.978374 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7598895f-d50c-4346-ab68-45e7b9db5ffd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:53:38.979933 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7598895f-d50c-4346-ab68-45e7b9db5ffd service placement] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:53:38.989771 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 763/1521] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:53:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:53:40.141629 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782c2fab-9bb1-416a-b217-cf0cd2b7471b None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:53:40.142230 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782c2fab-9bb1-416a-b217-cf0cd2b7471b None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:53:40.142230 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782c2fab-9bb1-416a-b217-cf0cd2b7471b None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:53:40.143499 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 759/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 08:53:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:53:40.148396 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bb92d6-2fc7-46f8-ae50-3b47c5994191 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:53:40.149408 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bb92d6-2fc7-46f8-ae50-3b47c5994191 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:53:40.149658 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bb92d6-2fc7-46f8-ae50-3b47c5994191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:53:40.172906 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-79bb92d6-2fc7-46f8-ae50-3b47c5994191 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:53:40.172906 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-79bb92d6-2fc7-46f8-ae50-3b47c5994191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:53:40.179822 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 764/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 08:53:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:53:40.333445 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782c2fab-9bb1-416a-b217-cf0cd2b7471b None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:53:40.348179 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782c2fab-9bb1-416a-b217-cf0cd2b7471b None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:53:40.354126 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092f5a8d-b08a-47cb-9586-59078c5abb49 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17da357e555420799ee681bffbcc268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:53:40.354716 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092f5a8d-b08a-47cb-9586-59078c5abb49 service nova] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:53:40.354995 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092f5a8d-b08a-47cb-9586-59078c5abb49 service nova] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:53:40.355265 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092f5a8d-b08a-47cb-9586-59078c5abb49 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:53:40.363397 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092f5a8d-b08a-47cb-9586-59078c5abb49 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:53:40.365024 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092f5a8d-b08a-47cb-9586-59078c5abb49 service nova] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:53:40.375143 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 760/1524] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:53:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:53:41.195636 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f223cd0-f13a-48c6-8ad1-587ca58a95f0 None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:53:41.196264 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f223cd0-f13a-48c6-8ad1-587ca58a95f0 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:53:41.196264 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f223cd0-f13a-48c6-8ad1-587ca58a95f0 None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:53:41.198152 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 765/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 08:53:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:53:41.202461 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1b4398-a8e0-4dbb-80cf-522016e58e2a None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:53:41.202905 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1b4398-a8e0-4dbb-80cf-522016e58e2a None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:53:41.203038 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1b4398-a8e0-4dbb-80cf-522016e58e2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:53:41.222139 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-cc1b4398-a8e0-4dbb-80cf-522016e58e2a None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:53:41.223399 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-cc1b4398-a8e0-4dbb-80cf-522016e58e2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:53:41.234554 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 761/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 08:53:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:53:41.391685 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f223cd0-f13a-48c6-8ad1-587ca58a95f0 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:53:41.407020 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f223cd0-f13a-48c6-8ad1-587ca58a95f0 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:53:41.412410 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03baa4cc-c8f0-4ce6-a74c-65bbc59b895b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17da357e555420799ee681bffbcc268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:53:41.413025 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03baa4cc-c8f0-4ce6-a74c-65bbc59b895b service nova] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:53:41.413304 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03baa4cc-c8f0-4ce6-a74c-65bbc59b895b service nova] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:53:41.413575 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03baa4cc-c8f0-4ce6-a74c-65bbc59b895b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:53:41.422241 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03baa4cc-c8f0-4ce6-a74c-65bbc59b895b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:53:41.423809 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03baa4cc-c8f0-4ce6-a74c-65bbc59b895b service nova] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:53:41.433098 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 766/1527] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:53:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:53:45.884851 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe2f789-5fb5-4dc0-8760-a263296dd730 None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:53:45.885409 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe2f789-5fb5-4dc0-8760-a263296dd730 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:53:45.885409 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe2f789-5fb5-4dc0-8760-a263296dd730 None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:53:45.886766 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 762/1528] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 08:53:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:53:45.892170 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afff3240-5e6b-4085-b1e3-b3bc41563abe None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:53:45.892668 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afff3240-5e6b-4085-b1e3-b3bc41563abe None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:53:45.892775 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afff3240-5e6b-4085-b1e3-b3bc41563abe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:53:45.913715 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-afff3240-5e6b-4085-b1e3-b3bc41563abe None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:53:45.914919 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-afff3240-5e6b-4085-b1e3-b3bc41563abe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:53:45.922779 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 767/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 08:53:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:53:46.214347 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe2f789-5fb5-4dc0-8760-a263296dd730 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:53:46.216034 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-5fe2f789-5fb5-4dc0-8760-a263296dd730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:53:46.297525 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe2f789-5fb5-4dc0-8760-a263296dd730 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:53:46.301183 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808b394f-38b5-47c7-9aa6-9e767b86a4f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:53:46.301779 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808b394f-38b5-47c7-9aa6-9e767b86a4f3 service neutron] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:53:46.302057 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808b394f-38b5-47c7-9aa6-9e767b86a4f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:53:46.302327 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808b394f-38b5-47c7-9aa6-9e767b86a4f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:53:46.311654 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808b394f-38b5-47c7-9aa6-9e767b86a4f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:53:46.313109 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808b394f-38b5-47c7-9aa6-9e767b86a4f3 service neutron] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:53:46.322429 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 763/1530] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:53:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:53:48.214344 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4d778b-0068-445f-af94-2aa34616faf9 None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:53:48.214873 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4d778b-0068-445f-af94-2aa34616faf9 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:53:48.214873 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4d778b-0068-445f-af94-2aa34616faf9 None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:53:48.216232 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 768/1531] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 08:53:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:53:48.221335 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2489c1e0-11b4-4da2-acf0-9e1a99be2c77 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:53:48.221763 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2489c1e0-11b4-4da2-acf0-9e1a99be2c77 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:53:48.221918 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2489c1e0-11b4-4da2-acf0-9e1a99be2c77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:53:48.240922 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-2489c1e0-11b4-4da2-acf0-9e1a99be2c77 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:53:48.242108 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-2489c1e0-11b4-4da2-acf0-9e1a99be2c77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:53:48.250286 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 764/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 08:53:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:53:48.533882 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4d778b-0068-445f-af94-2aa34616faf9 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:53:48.545113 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4d778b-0068-445f-af94-2aa34616faf9 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:53:48.548580 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701e0e59-31fe-4ac7-aaab-d3c378bdffb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:53:48.549173 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701e0e59-31fe-4ac7-aaab-d3c378bdffb4 service neutron] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:53:48.549451 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701e0e59-31fe-4ac7-aaab-d3c378bdffb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:53:48.549724 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701e0e59-31fe-4ac7-aaab-d3c378bdffb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:53:48.559093 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701e0e59-31fe-4ac7-aaab-d3c378bdffb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:53:48.560522 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701e0e59-31fe-4ac7-aaab-d3c378bdffb4 service neutron] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:53:48.569837 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 769/1533] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:53:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:53:51.842087 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392fc377-517c-48c7-bf48-f3ebcb53614d None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:53:51.842680 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392fc377-517c-48c7-bf48-f3ebcb53614d None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:53:51.842680 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392fc377-517c-48c7-bf48-f3ebcb53614d None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:53:51.843975 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 765/1534] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 08:53:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:53:51.849172 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa0816f-2524-497d-8a74-be47ac6d8f2f None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:53:51.849579 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa0816f-2524-497d-8a74-be47ac6d8f2f None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:53:51.849727 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa0816f-2524-497d-8a74-be47ac6d8f2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:53:51.869422 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-3aa0816f-2524-497d-8a74-be47ac6d8f2f None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:53:51.870631 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-3aa0816f-2524-497d-8a74-be47ac6d8f2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:53:51.878412 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 770/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 08:53:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:53:52.433482 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392fc377-517c-48c7-bf48-f3ebcb53614d None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:53:52.435187 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-392fc377-517c-48c7-bf48-f3ebcb53614d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:53:52.499978 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:53:52.513209 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392fc377-517c-48c7-bf48-f3ebcb53614d None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:53:52.516896 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2954d003-aed1-45fe-89a8-05dbad8adbe9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbb9ac667f04c35a63b2689df18efed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:53:52.517493 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2954d003-aed1-45fe-89a8-05dbad8adbe9 service cinder] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:53:52.517766 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2954d003-aed1-45fe-89a8-05dbad8adbe9 service cinder] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:53:52.518033 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2954d003-aed1-45fe-89a8-05dbad8adbe9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:53:52.527925 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2954d003-aed1-45fe-89a8-05dbad8adbe9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:53:52.529375 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2954d003-aed1-45fe-89a8-05dbad8adbe9 service cinder] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:53:52.540531 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 766/1536] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:53:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:53:54.464650 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5e6c3e-bc76-4800-84f3-43a12dc18efb None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:53:54.465158 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5e6c3e-bc76-4800-84f3-43a12dc18efb None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:53:54.465158 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5e6c3e-bc76-4800-84f3-43a12dc18efb None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:53:54.466453 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 771/1537] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 08:53:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:53:54.471826 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412acf3d-89f8-443d-a143-fe70d8355454 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:53:54.472216 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412acf3d-89f8-443d-a143-fe70d8355454 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:53:54.472368 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412acf3d-89f8-443d-a143-fe70d8355454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:53:54.491495 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-412acf3d-89f8-443d-a143-fe70d8355454 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:53:54.492782 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-412acf3d-89f8-443d-a143-fe70d8355454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:53:54.500589 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 767/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 08:53:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:53:55.051083 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5e6c3e-bc76-4800-84f3-43a12dc18efb None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:53:55.052818 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-dd5e6c3e-bc76-4800-84f3-43a12dc18efb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:53:55.066985 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5e6c3e-bc76-4800-84f3-43a12dc18efb None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:53:55.072267 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04032942-b40f-4270-96ef-430b3797665d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbb9ac667f04c35a63b2689df18efed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:53:55.072267 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04032942-b40f-4270-96ef-430b3797665d service cinder] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:53:55.072267 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04032942-b40f-4270-96ef-430b3797665d service cinder] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:53:55.072717 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04032942-b40f-4270-96ef-430b3797665d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:53:55.082180 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04032942-b40f-4270-96ef-430b3797665d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:53:55.083665 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04032942-b40f-4270-96ef-430b3797665d service cinder] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:53:55.093149 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 772/1539] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:53:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:54:04.486640 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=11 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:54:05.088882 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=33 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:22.448975 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bba8c6-9c9c-4106-ac7e-97e9dd483520 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:22.449865 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bba8c6-9c9c-4106-ac7e-97e9dd483520 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:22.449865 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bba8c6-9c9c-4106-ac7e-97e9dd483520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:22.476451 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-55bba8c6-9c9c-4106-ac7e-97e9dd483520 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:22.477603 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-55bba8c6-9c9c-4106-ac7e-97e9dd483520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:22.489320 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 768/1540] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:55:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:22.498995 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599d96d5-6dad-4280-beb0-607ff2024759 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:22.499802 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599d96d5-6dad-4280-beb0-607ff2024759 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:22.499802 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599d96d5-6dad-4280-beb0-607ff2024759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:22.537549 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-599d96d5-6dad-4280-beb0-607ff2024759 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:22.538759 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-599d96d5-6dad-4280-beb0-607ff2024759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:22.547258 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 773/1541] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:55:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:22.552253 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bba8c6-9c9c-4106-ac7e-97e9dd483520 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:22.563343 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bba8c6-9c9c-4106-ac7e-97e9dd483520 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:22.579447 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f895d535-5e8f-4ce0-b29a-9fee94b2418b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:22.580654 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f895d535-5e8f-4ce0-b29a-9fee94b2418b admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:22.580654 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f895d535-5e8f-4ce0-b29a-9fee94b2418b admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:22.580970 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f895d535-5e8f-4ce0-b29a-9fee94b2418b admin admin] PATH_INFO: `/v3/domains` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:22.581557 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f895d535-5e8f-4ce0-b29a-9fee94b2418b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:55:22.581860 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f895d535-5e8f-4ce0-b29a-9fee94b2418b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:22.583471 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f895d535-5e8f-4ce0-b29a-9fee94b2418b admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:22.601215 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 769/1542] 10.222.0.55 () {58 vars in 1153 bytes} [Wed Aug 24 08:55:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:55:22.609880 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599d96d5-6dad-4280-beb0-607ff2024759 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:22.620131 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599d96d5-6dad-4280-beb0-607ff2024759 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:22.629082 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de424c6a-b507-495a-8725-69ba8bb8fb29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:22.629703 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de424c6a-b507-495a-8725-69ba8bb8fb29 admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:22.629981 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de424c6a-b507-495a-8725-69ba8bb8fb29 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:22.630260 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de424c6a-b507-495a-8725-69ba8bb8fb29 admin admin] PATH_INFO: `/v3/projects` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:22.630866 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de424c6a-b507-495a-8725-69ba8bb8fb29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:22.636846 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de424c6a-b507-495a-8725-69ba8bb8fb29 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:22.754472 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d885fd-0ece-4e88-b8d7-889ef3119430 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:22.754472 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d885fd-0ece-4e88-b8d7-889ef3119430 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:22.754472 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d885fd-0ece-4e88-b8d7-889ef3119430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:22.776761 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-f5d885fd-0ece-4e88-b8d7-889ef3119430 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:22.778132 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-f5d885fd-0ece-4e88-b8d7-889ef3119430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:22.791702 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 770/1543] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:55:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:55:22.800852 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d54aad-d99e-49a0-b44c-f2308b26b308 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:22.801194 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d54aad-d99e-49a0-b44c-f2308b26b308 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:22.801389 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d54aad-d99e-49a0-b44c-f2308b26b308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:22.826337 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-57d54aad-d99e-49a0-b44c-f2308b26b308 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:22.827574 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-57d54aad-d99e-49a0-b44c-f2308b26b308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:22.839998 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 771/1544] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:55:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:55:22.844870 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d54aad-d99e-49a0-b44c-f2308b26b308 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:22.855949 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d54aad-d99e-49a0-b44c-f2308b26b308 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:22.864799 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d997b2-bbcd-4db1-8119-5b4d79b8f814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:22.865651 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d997b2-bbcd-4db1-8119-5b4d79b8f814 admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:22.865926 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d997b2-bbcd-4db1-8119-5b4d79b8f814 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:22.866208 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d997b2-bbcd-4db1-8119-5b4d79b8f814 admin admin] PATH_INFO: `/v3/domains` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:22.866904 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d997b2-bbcd-4db1-8119-5b4d79b8f814 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:55:22.867213 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d997b2-bbcd-4db1-8119-5b4d79b8f814 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:22.868910 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d997b2-bbcd-4db1-8119-5b4d79b8f814 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:22.883413 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 774/1545] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 08:55:22 2022] POST /identity/v3/projects => generated 375 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:55:22.886121 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 772/1546] 10.222.0.55 () {58 vars in 1153 bytes} [Wed Aug 24 08:55:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:55:22.894932 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de424c6a-b507-495a-8725-69ba8bb8fb29 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:22.898931 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d997b2-bbcd-4db1-8119-5b4d79b8f814 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:22.910061 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d997b2-bbcd-4db1-8119-5b4d79b8f814 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:22.979467 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05885131-5dbc-4f23-90c0-a92aeb167b07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:22.980091 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05885131-5dbc-4f23-90c0-a92aeb167b07 admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:22.980369 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05885131-5dbc-4f23-90c0-a92aeb167b07 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:22.980675 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05885131-5dbc-4f23-90c0-a92aeb167b07 admin admin] PATH_INFO: `/v3/users` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:22.981383 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05885131-5dbc-4f23-90c0-a92aeb167b07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:22.983450 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05885131-5dbc-4f23-90c0-a92aeb167b07 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:23.010906 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de424c6a-b507-495a-8725-69ba8bb8fb29 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:23.018187 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d62aca9-ba90-4bff-9c66-5a3f3971091b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:23.018991 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d62aca9-ba90-4bff-9c66-5a3f3971091b admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:23.019282 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d62aca9-ba90-4bff-9c66-5a3f3971091b admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:23.019549 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d62aca9-ba90-4bff-9c66-5a3f3971091b admin admin] PATH_INFO: `/v3/projects` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:23.020192 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d62aca9-ba90-4bff-9c66-5a3f3971091b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:23.022265 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d62aca9-ba90-4bff-9c66-5a3f3971091b admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:23.072352 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 773/1547] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 08:55:22 2022] POST /identity/v3/users => generated 288 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:55:23.073520 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fcf334-b00c-41a8-a9a9-7d0d2c9d5e24 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:23.073801 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fcf334-b00c-41a8-a9a9-7d0d2c9d5e24 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:23.074118 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fcf334-b00c-41a8-a9a9-7d0d2c9d5e24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:23.102449 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-58fcf334-b00c-41a8-a9a9-7d0d2c9d5e24 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:23.103701 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-58fcf334-b00c-41a8-a9a9-7d0d2c9d5e24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:23.117150 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 774/1548] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:55:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:23.118235 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fcf334-b00c-41a8-a9a9-7d0d2c9d5e24 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:23.133116 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fcf334-b00c-41a8-a9a9-7d0d2c9d5e24 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:23.138009 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1b4e35-4889-4c93-9c9c-b3aba8f2901a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:23.138645 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1b4e35-4889-4c93-9c9c-b3aba8f2901a admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:23.138926 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1b4e35-4889-4c93-9c9c-b3aba8f2901a admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:23.139212 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1b4e35-4889-4c93-9c9c-b3aba8f2901a admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:23.139872 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1b4e35-4889-4c93-9c9c-b3aba8f2901a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:23.141905 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1b4e35-4889-4c93-9c9c-b3aba8f2901a admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:23.156926 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 775/1549] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 08:55:22 2022] POST /identity/v3/projects => generated 377 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:55:23.158465 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82926ef-d064-4dc2-b649-ef489d9019a2 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:23.158754 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82926ef-d064-4dc2-b649-ef489d9019a2 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:23.159031 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82926ef-d064-4dc2-b649-ef489d9019a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:23.175553 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 775/1550] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:55:23 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:55:23.179206 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1b4e35-4889-4c93-9c9c-b3aba8f2901a admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:23.204702 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1b4e35-4889-4c93-9c9c-b3aba8f2901a admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:23.221508 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-a82926ef-d064-4dc2-b649-ef489d9019a2 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:23.222970 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-a82926ef-d064-4dc2-b649-ef489d9019a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:23.345227 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a6d364-312d-4ae3-a134-d615a863a5c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:23.345856 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a6d364-312d-4ae3-a134-d615a863a5c3 admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:23.346142 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a6d364-312d-4ae3-a134-d615a863a5c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:23.346430 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a6d364-312d-4ae3-a134-d615a863a5c3 admin admin] PATH_INFO: `/v3/users` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:23.347050 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a6d364-312d-4ae3-a134-d615a863a5c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:23.352648 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a6d364-312d-4ae3-a134-d615a863a5c3 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:23.431337 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 776/1551] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:55:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:23.433351 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82926ef-d064-4dc2-b649-ef489d9019a2 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:23.462340 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82926ef-d064-4dc2-b649-ef489d9019a2 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:23.476986 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0cc8db-513c-49d1-9e8c-b9089ff88dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:23.478480 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0cc8db-513c-49d1-9e8c-b9089ff88dac admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:23.479285 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0cc8db-513c-49d1-9e8c-b9089ff88dac admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:23.480071 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0cc8db-513c-49d1-9e8c-b9089ff88dac admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:23.481992 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0cc8db-513c-49d1-9e8c-b9089ff88dac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:23.485613 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0cc8db-513c-49d1-9e8c-b9089ff88dac admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:23.516287 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 777/1552] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:55:23 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:55:23.518039 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0cc8db-513c-49d1-9e8c-b9089ff88dac admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:23.518713 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 776/1553] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 08:55:23 2022] POST /identity/v3/users => generated 289 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:55:23.525371 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b874e971-b6ca-412c-9505-0120d7ea7bac None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:23.526963 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b874e971-b6ca-412c-9505-0120d7ea7bac None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:23.526963 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b874e971-b6ca-412c-9505-0120d7ea7bac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:23.544205 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0cc8db-513c-49d1-9e8c-b9089ff88dac admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:23.559590 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c86cde-f41f-4d23-901a-a64891446849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:23.560229 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c86cde-f41f-4d23-901a-a64891446849 admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:23.560510 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c86cde-f41f-4d23-901a-a64891446849 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:23.560802 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c86cde-f41f-4d23-901a-a64891446849 admin admin] PATH_INFO: `/v3/domains` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:23.562894 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c86cde-f41f-4d23-901a-a64891446849 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:55:23.563211 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c86cde-f41f-4d23-901a-a64891446849 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:23.564236 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: WARNING py.warnings [None req-35c86cde-f41f-4d23-901a-a64891446849 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 08:55:23.564236 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: warnings.warn(msg) Aug 24 08:55:23.564236 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: Aug 24 08:55:23.565158 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c86cde-f41f-4d23-901a-a64891446849 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:23.572338 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-b874e971-b6ca-412c-9505-0120d7ea7bac None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:23.573616 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-b874e971-b6ca-412c-9505-0120d7ea7bac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:23.589517 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 778/1554] 10.222.0.55 () {58 vars in 1153 bytes} [Wed Aug 24 08:55:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:55:23.590363 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c86cde-f41f-4d23-901a-a64891446849 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:23.600215 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 777/1555] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:55:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:23.601182 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b874e971-b6ca-412c-9505-0120d7ea7bac None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:23.618175 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b874e971-b6ca-412c-9505-0120d7ea7bac None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:23.624457 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c86cde-f41f-4d23-901a-a64891446849 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:23.627782 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98c8973-cd95-4fff-90d7-9741d10dea8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:23.628405 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98c8973-cd95-4fff-90d7-9741d10dea8b admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:23.628765 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98c8973-cd95-4fff-90d7-9741d10dea8b admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:23.629035 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98c8973-cd95-4fff-90d7-9741d10dea8b admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:23.629615 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98c8973-cd95-4fff-90d7-9741d10dea8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:23.633269 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98c8973-cd95-4fff-90d7-9741d10dea8b admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:23.637832 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b3dc53-9693-4466-9795-2ebe2986662d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:23.638522 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b3dc53-9693-4466-9795-2ebe2986662d admin admin] REQUEST_METHOD: `PUT` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:23.638792 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b3dc53-9693-4466-9795-2ebe2986662d admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:23.639249 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b3dc53-9693-4466-9795-2ebe2986662d admin admin] PATH_INFO: `/v3/projects/460e817ed5584689a98e649af18c4451/users/5e8fd689c9af47ffac97e81255fc04f9/roles/b674fd0a592e4e0cada811bba5ceea4d` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:23.641533 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:23.658721 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 778/1556] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:55:23 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:55:23.660222 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6984c8-4a0a-47f4-8fea-8835969039cc None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:23.660509 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6984c8-4a0a-47f4-8fea-8835969039cc None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:23.664103 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6984c8-4a0a-47f4-8fea-8835969039cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:23.670743 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b3dc53-9693-4466-9795-2ebe2986662d admin admin] RBAC: Authorizing `identity:create_grant(project_id=460e817ed5584689a98e649af18c4451, user_id=5e8fd689c9af47ffac97e81255fc04f9, role_id=b674fd0a592e4e0cada811bba5ceea4d)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:23.676651 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b3dc53-9693-4466-9795-2ebe2986662d admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:23.707763 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-2f6984c8-4a0a-47f4-8fea-8835969039cc None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:23.709042 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-2f6984c8-4a0a-47f4-8fea-8835969039cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:23.722151 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 779/1557] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:55:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:23.723139 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6984c8-4a0a-47f4-8fea-8835969039cc None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:23.733437 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6984c8-4a0a-47f4-8fea-8835969039cc None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:23.738478 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffca02a-21c5-478c-b3c8-6ccc52097e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:23.739122 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffca02a-21c5-478c-b3c8-6ccc52097e2e admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:23.739402 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffca02a-21c5-478c-b3c8-6ccc52097e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:23.739673 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffca02a-21c5-478c-b3c8-6ccc52097e2e admin admin] PATH_INFO: `/v3/projects` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:23.740331 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffca02a-21c5-478c-b3c8-6ccc52097e2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:23.742092 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: WARNING py.warnings [None req-8ffca02a-21c5-478c-b3c8-6ccc52097e2e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 08:55:23.742092 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: warnings.warn(msg) Aug 24 08:55:23.742092 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: Aug 24 08:55:23.744148 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffca02a-21c5-478c-b3c8-6ccc52097e2e admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:23.775512 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:23.778467 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 779/1558] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 08:55:23 2022] PUT /identity/v3/projects/460e817ed5584689a98e649af18c4451/users/5e8fd689c9af47ffac97e81255fc04f9/roles/b674fd0a592e4e0cada811bba5ceea4d => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:55:23.785797 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fe6ef7-d47a-45b1-9382-88dfcfa0fc07 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:23.786289 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fe6ef7-d47a-45b1-9382-88dfcfa0fc07 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:23.786460 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fe6ef7-d47a-45b1-9382-88dfcfa0fc07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:23.812455 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-d2fe6ef7-d47a-45b1-9382-88dfcfa0fc07 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:23.813772 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-d2fe6ef7-d47a-45b1-9382-88dfcfa0fc07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:23.866963 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 780/1559] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 08:55:23 2022] POST /identity/v3/projects => generated 383 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:55:23.867918 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffca02a-21c5-478c-b3c8-6ccc52097e2e admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:23.902240 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 780/1560] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:55:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:23.907382 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb7eb5b-c6e3-4a25-9ff5-18b49ccd004e None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:23.907808 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb7eb5b-c6e3-4a25-9ff5-18b49ccd004e None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:23.908090 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb7eb5b-c6e3-4a25-9ff5-18b49ccd004e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:23.929576 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-ddb7eb5b-c6e3-4a25-9ff5-18b49ccd004e None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:23.931003 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-ddb7eb5b-c6e3-4a25-9ff5-18b49ccd004e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:23.960984 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffca02a-21c5-478c-b3c8-6ccc52097e2e admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:23.967420 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09a1343-b8c6-4a2d-9ad3-f911f451b4d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:23.968370 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09a1343-b8c6-4a2d-9ad3-f911f451b4d6 admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:23.968688 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09a1343-b8c6-4a2d-9ad3-f911f451b4d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:23.969600 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09a1343-b8c6-4a2d-9ad3-f911f451b4d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:23.970894 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09a1343-b8c6-4a2d-9ad3-f911f451b4d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:23.973618 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09a1343-b8c6-4a2d-9ad3-f911f451b4d6 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:23.986078 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 781/1561] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:55:23 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:55:23.989008 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd109bb-ed3b-4a9a-b023-2cd8be830ed2 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:23.989460 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd109bb-ed3b-4a9a-b023-2cd8be830ed2 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:23.989752 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd109bb-ed3b-4a9a-b023-2cd8be830ed2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:24.011621 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 781/1562] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:55:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:24.013056 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7398b54e-1ea4-41db-97fd-ca9a6bc8707f None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:24.013346 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7398b54e-1ea4-41db-97fd-ca9a6bc8707f None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:24.013652 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7398b54e-1ea4-41db-97fd-ca9a6bc8707f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:24.035581 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-7398b54e-1ea4-41db-97fd-ca9a6bc8707f None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:24.036841 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-7398b54e-1ea4-41db-97fd-ca9a6bc8707f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:24.043727 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-1cd109bb-ed3b-4a9a-b023-2cd8be830ed2 None None] MFA Rules not processed for user `5e8fd689c9af47ffac97e81255fc04f9`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:24.045869 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-1cd109bb-ed3b-4a9a-b023-2cd8be830ed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:24.054756 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 782/1563] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:55:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:24.059660 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7398b54e-1ea4-41db-97fd-ca9a6bc8707f None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:24.066297 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7398b54e-1ea4-41db-97fd-ca9a6bc8707f None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:24.072449 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e9f593-bcb1-4ccd-937e-76130bb1a7f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:24.073247 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e9f593-bcb1-4ccd-937e-76130bb1a7f7 admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:24.073525 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e9f593-bcb1-4ccd-937e-76130bb1a7f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:24.073791 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e9f593-bcb1-4ccd-937e-76130bb1a7f7 admin admin] PATH_INFO: `/v3/users` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:24.074418 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e9f593-bcb1-4ccd-937e-76130bb1a7f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:24.075427 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: WARNING py.warnings [None req-36e9f593-bcb1-4ccd-937e-76130bb1a7f7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 08:55:24.075427 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: warnings.warn(msg) Aug 24 08:55:24.075427 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: Aug 24 08:55:24.076666 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e9f593-bcb1-4ccd-937e-76130bb1a7f7 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:24.126235 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 782/1564] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:55:23 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:24.127043 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd109bb-ed3b-4a9a-b023-2cd8be830ed2 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:24.138604 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd109bb-ed3b-4a9a-b023-2cd8be830ed2 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:24.143368 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010174b1-0e39-463d-9137-e78c56b093db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:24.143979 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010174b1-0e39-463d-9137-e78c56b093db admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:24.144409 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010174b1-0e39-463d-9137-e78c56b093db admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:24.144764 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010174b1-0e39-463d-9137-e78c56b093db admin admin] PATH_INFO: `/v3/domains` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:24.145484 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010174b1-0e39-463d-9137-e78c56b093db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:55:24.145882 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010174b1-0e39-463d-9137-e78c56b093db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:24.147531 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010174b1-0e39-463d-9137-e78c56b093db admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:24.160587 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 783/1565] 10.222.0.55 () {58 vars in 1153 bytes} [Wed Aug 24 08:55:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:55:24.168242 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010174b1-0e39-463d-9137-e78c56b093db admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:24.172231 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010174b1-0e39-463d-9137-e78c56b093db admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:24.177711 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 783/1566] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 08:55:24 2022] POST /identity/v3/users => generated 292 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:55:24.177711 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02b22ee-3e20-479b-a887-38f91133771d None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:24.177711 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02b22ee-3e20-479b-a887-38f91133771d None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:24.177711 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02b22ee-3e20-479b-a887-38f91133771d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:24.178713 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a899b80-a779-49bf-993a-bd482d42671b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:24.178713 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a899b80-a779-49bf-993a-bd482d42671b admin admin] REQUEST_METHOD: `PUT` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:24.178713 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a899b80-a779-49bf-993a-bd482d42671b admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:24.178986 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a899b80-a779-49bf-993a-bd482d42671b admin admin] PATH_INFO: `/v3/projects/d0c61eee777c46779632c08ee11946c5/users/718a0dfdef714e358a74ad8e424b1b8a/roles/b674fd0a592e4e0cada811bba5ceea4d` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:24.195986 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a899b80-a779-49bf-993a-bd482d42671b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0c61eee777c46779632c08ee11946c5, user_id=718a0dfdef714e358a74ad8e424b1b8a, role_id=b674fd0a592e4e0cada811bba5ceea4d)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:24.197070 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: WARNING py.warnings [None req-0a899b80-a779-49bf-993a-bd482d42671b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 08:55:24.197070 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: warnings.warn(msg) Aug 24 08:55:24.197070 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: Aug 24 08:55:24.203973 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-a02b22ee-3e20-479b-a887-38f91133771d None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:24.204154 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a899b80-a779-49bf-993a-bd482d42671b admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:24.204258 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-a02b22ee-3e20-479b-a887-38f91133771d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:24.208308 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 784/1567] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:55:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:24.209444 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02b22ee-3e20-479b-a887-38f91133771d None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:24.219476 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02b22ee-3e20-479b-a887-38f91133771d None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:24.224203 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500e013a-a5e5-4af4-a43b-ebdd700ac323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:24.225714 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e013a-a5e5-4af4-a43b-ebdd700ac323 admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:24.225714 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e013a-a5e5-4af4-a43b-ebdd700ac323 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:24.225714 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e013a-a5e5-4af4-a43b-ebdd700ac323 admin admin] PATH_INFO: `/v3/domains` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:24.226087 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500e013a-a5e5-4af4-a43b-ebdd700ac323 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:55:24.226386 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500e013a-a5e5-4af4-a43b-ebdd700ac323 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:24.227983 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500e013a-a5e5-4af4-a43b-ebdd700ac323 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:24.241082 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 785/1568] 10.222.0.55 () {58 vars in 1153 bytes} [Wed Aug 24 08:55:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:55:24.250146 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83927d0-1dee-4581-a943-7238f1c1ad97 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:24.250146 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83927d0-1dee-4581-a943-7238f1c1ad97 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:24.250146 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83927d0-1dee-4581-a943-7238f1c1ad97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:24.266087 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-a83927d0-1dee-4581-a943-7238f1c1ad97 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:24.267313 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-a83927d0-1dee-4581-a943-7238f1c1ad97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:24.275575 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 786/1569] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:55:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:24.277145 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3370459-040e-4e88-86d3-086a459802be None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:24.277318 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3370459-040e-4e88-86d3-086a459802be None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:24.277575 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3370459-040e-4e88-86d3-086a459802be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:24.296984 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-d3370459-040e-4e88-86d3-086a459802be None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:24.298175 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-d3370459-040e-4e88-86d3-086a459802be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:24.306590 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 787/1570] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:55:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:24.307796 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a30bc7-0fd3-463f-8345-9116ba94776b None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:24.308078 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a30bc7-0fd3-463f-8345-9116ba94776b None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:24.308342 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a30bc7-0fd3-463f-8345-9116ba94776b None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:24.309664 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 788/1571] 10.222.0.55 () {54 vars in 962 bytes} [Wed Aug 24 08:55:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:55:24.310332 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a30bc7-0fd3-463f-8345-9116ba94776b None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:24.323704 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a30bc7-0fd3-463f-8345-9116ba94776b None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:24.334270 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae19389c-6d67-4499-b1b4-dee95e914af0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:24.334270 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae19389c-6d67-4499-b1b4-dee95e914af0 admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:24.334270 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae19389c-6d67-4499-b1b4-dee95e914af0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:24.334270 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae19389c-6d67-4499-b1b4-dee95e914af0 admin admin] PATH_INFO: `/v3/projects` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:24.334674 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae19389c-6d67-4499-b1b4-dee95e914af0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:24.337421 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae19389c-6d67-4499-b1b4-dee95e914af0 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:24.365092 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 784/1572] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 08:55:24 2022] PUT /identity/v3/projects/d0c61eee777c46779632c08ee11946c5/users/718a0dfdef714e358a74ad8e424b1b8a/roles/b674fd0a592e4e0cada811bba5ceea4d => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:55:24.366682 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a899b80-a779-49bf-993a-bd482d42671b admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:24.388192 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:24.432347 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a899b80-a779-49bf-993a-bd482d42671b admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:24.437072 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d66074d-58ca-435b-9de5-502280848fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:24.437678 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d66074d-58ca-435b-9de5-502280848fdd admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:24.437959 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d66074d-58ca-435b-9de5-502280848fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:24.438231 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d66074d-58ca-435b-9de5-502280848fdd admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:24.438873 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d66074d-58ca-435b-9de5-502280848fdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:24.440472 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d66074d-58ca-435b-9de5-502280848fdd admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:24.453640 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 785/1573] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:55:24 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:55:24.453640 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1f99ad-1978-4f18-9a3b-575a0a37f9c6 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:24.453640 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1f99ad-1978-4f18-9a3b-575a0a37f9c6 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:24.453640 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1f99ad-1978-4f18-9a3b-575a0a37f9c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:24.455983 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 789/1574] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 08:55:24 2022] POST /identity/v3/projects => generated 405 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:55:24.456861 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae19389c-6d67-4499-b1b4-dee95e914af0 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:24.472209 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-bf1f99ad-1978-4f18-9a3b-575a0a37f9c6 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:24.473371 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-bf1f99ad-1978-4f18-9a3b-575a0a37f9c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:24.483837 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:24.523583 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae19389c-6d67-4499-b1b4-dee95e914af0 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:24.528808 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa359f8-aa42-4435-9d8c-d0066e32ceb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:24.529435 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa359f8-aa42-4435-9d8c-d0066e32ceb9 admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:24.529720 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa359f8-aa42-4435-9d8c-d0066e32ceb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:24.529990 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa359f8-aa42-4435-9d8c-d0066e32ceb9 admin admin] PATH_INFO: `/v3/projects` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:24.530626 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa359f8-aa42-4435-9d8c-d0066e32ceb9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:24.532558 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa359f8-aa42-4435-9d8c-d0066e32ceb9 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:24.542520 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 786/1575] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:55:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:24.543924 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eb3c11-36c3-4a87-838c-cc35312e2c81 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:24.544272 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eb3c11-36c3-4a87-838c-cc35312e2c81 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:24.545350 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eb3c11-36c3-4a87-838c-cc35312e2c81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:24.565135 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-d7eb3c11-36c3-4a87-838c-cc35312e2c81 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:24.566352 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-d7eb3c11-36c3-4a87-838c-cc35312e2c81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:24.574397 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 787/1576] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:55:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:24.576061 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7eb3c11-36c3-4a87-838c-cc35312e2c81 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:24.585562 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7eb3c11-36c3-4a87-838c-cc35312e2c81 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:24.590218 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a590df-303f-418e-816c-8294fe128a14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:24.590799 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a590df-303f-418e-816c-8294fe128a14 admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:24.591078 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a590df-303f-418e-816c-8294fe128a14 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:24.591349 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a590df-303f-418e-816c-8294fe128a14 admin admin] PATH_INFO: `/v3/domains` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:24.592108 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a590df-303f-418e-816c-8294fe128a14 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:55:24.592298 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a590df-303f-418e-816c-8294fe128a14 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:24.593928 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a590df-303f-418e-816c-8294fe128a14 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:24.606049 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 788/1577] 10.222.0.55 () {58 vars in 1153 bytes} [Wed Aug 24 08:55:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:55:24.607322 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa290c2-28a1-4e95-a590-14e5e77e093e None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:24.607629 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa290c2-28a1-4e95-a590-14e5e77e093e None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:24.607906 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa290c2-28a1-4e95-a590-14e5e77e093e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:24.646045 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-7fa290c2-28a1-4e95-a590-14e5e77e093e None None] MFA Rules not processed for user `f17da357e555420799ee681bffbcc268`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:24.647187 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-7fa290c2-28a1-4e95-a590-14e5e77e093e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:24.658325 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 790/1578] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 08:55:24 2022] POST /identity/v3/projects => generated 371 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:55:24.659652 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0a153c-b736-448a-95ba-c75fc7467f06 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:24.659940 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0a153c-b736-448a-95ba-c75fc7467f06 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:24.660274 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0a153c-b736-448a-95ba-c75fc7467f06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:24.681827 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-ee0a153c-b736-448a-95ba-c75fc7467f06 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:24.683098 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-ee0a153c-b736-448a-95ba-c75fc7467f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:24.719896 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 789/1579] 10.222.0.55 () {58 vars in 1059 bytes} [Wed Aug 24 08:55:24 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:24.720812 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa290c2-28a1-4e95-a590-14e5e77e093e None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:24.731795 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa290c2-28a1-4e95-a590-14e5e77e093e None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:24.742348 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:24.744208 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 791/1580] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:55:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:24.745089 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0a153c-b736-448a-95ba-c75fc7467f06 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:24.757278 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0a153c-b736-448a-95ba-c75fc7467f06 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:24.762214 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5d82b4-090a-4910-958a-994b21c8f09e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:24.762833 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5d82b4-090a-4910-958a-994b21c8f09e admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:24.763261 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5d82b4-090a-4910-958a-994b21c8f09e admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:24.763918 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5d82b4-090a-4910-958a-994b21c8f09e admin admin] PATH_INFO: `/v3/users` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:24.764204 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5d82b4-090a-4910-958a-994b21c8f09e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:24.766199 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5d82b4-090a-4910-958a-994b21c8f09e admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:24.791976 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086b370a-bd1d-420a-9bd3-9e5074db4177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:24.792608 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086b370a-bd1d-420a-9bd3-9e5074db4177 admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:24.792887 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086b370a-bd1d-420a-9bd3-9e5074db4177 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:24.793180 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086b370a-bd1d-420a-9bd3-9e5074db4177 admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:24.793812 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086b370a-bd1d-420a-9bd3-9e5074db4177 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:24.795403 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086b370a-bd1d-420a-9bd3-9e5074db4177 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:24.805489 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 790/1581] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:55:24 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:55:24.808598 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086b370a-bd1d-420a-9bd3-9e5074db4177 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:24.817427 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086b370a-bd1d-420a-9bd3-9e5074db4177 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:24.822305 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffbb153-726a-48f6-9a75-de9b3542386b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:24.822751 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffbb153-726a-48f6-9a75-de9b3542386b admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:24.823042 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffbb153-726a-48f6-9a75-de9b3542386b admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:24.823273 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffbb153-726a-48f6-9a75-de9b3542386b admin admin] PATH_INFO: `/v3/domains` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:24.824038 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffbb153-726a-48f6-9a75-de9b3542386b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:55:24.824261 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffbb153-726a-48f6-9a75-de9b3542386b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:24.825843 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffbb153-726a-48f6-9a75-de9b3542386b admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:24.839748 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 791/1582] 10.222.0.55 () {58 vars in 1153 bytes} [Wed Aug 24 08:55:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:55:24.840615 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffbb153-726a-48f6-9a75-de9b3542386b admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:24.851375 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffbb153-726a-48f6-9a75-de9b3542386b admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:24.856016 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a97144-9ff7-4280-8ca5-d0a27747f243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:24.856645 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a97144-9ff7-4280-8ca5-d0a27747f243 admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:24.856927 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a97144-9ff7-4280-8ca5-d0a27747f243 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:24.857201 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a97144-9ff7-4280-8ca5-d0a27747f243 admin admin] PATH_INFO: `/v3/domains` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:24.857866 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a97144-9ff7-4280-8ca5-d0a27747f243 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:55:24.858167 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a97144-9ff7-4280-8ca5-d0a27747f243 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:24.859755 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a97144-9ff7-4280-8ca5-d0a27747f243 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:24.871248 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 792/1583] 10.222.0.55 () {58 vars in 1153 bytes} [Wed Aug 24 08:55:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:55:24.872036 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a97144-9ff7-4280-8ca5-d0a27747f243 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:24.881555 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a97144-9ff7-4280-8ca5-d0a27747f243 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:24.886620 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5932465-a726-41e9-ba53-32b31c9db2a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:24.887207 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5932465-a726-41e9-ba53-32b31c9db2a7 admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:24.887486 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5932465-a726-41e9-ba53-32b31c9db2a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:24.887754 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5932465-a726-41e9-ba53-32b31c9db2a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:24.888370 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5932465-a726-41e9-ba53-32b31c9db2a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:24.890305 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5932465-a726-41e9-ba53-32b31c9db2a7 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:24.936776 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 792/1584] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 08:55:24 2022] POST /identity/v3/users => generated 303 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:55:24.937511 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5d82b4-090a-4910-958a-994b21c8f09e admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:24.947084 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5d82b4-090a-4910-958a-994b21c8f09e admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:24.951734 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3e68ab-5d9d-4fbe-a201-735d55661469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:24.952347 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3e68ab-5d9d-4fbe-a201-735d55661469 admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:24.952643 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3e68ab-5d9d-4fbe-a201-735d55661469 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:24.952921 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3e68ab-5d9d-4fbe-a201-735d55661469 admin admin] PATH_INFO: `/v3/users` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:24.953592 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3e68ab-5d9d-4fbe-a201-735d55661469 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:24.955436 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3e68ab-5d9d-4fbe-a201-735d55661469 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:25.041313 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 793/1585] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 08:55:24 2022] POST /identity/v3/projects => generated 391 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:55:25.042342 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5932465-a726-41e9-ba53-32b31c9db2a7 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:25.065555 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 793/1586] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 08:55:24 2022] POST /identity/v3/users => generated 286 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:55:25.066523 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3e68ab-5d9d-4fbe-a201-735d55661469 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:25.118256 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5932465-a726-41e9-ba53-32b31c9db2a7 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:25.124001 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e229c64f-fb37-4dda-b82a-3d44ccc3258f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17da357e555420799ee681bffbcc268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:25.124653 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e229c64f-fb37-4dda-b82a-3d44ccc3258f service nova] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:25.124937 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e229c64f-fb37-4dda-b82a-3d44ccc3258f service nova] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:25.125218 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e229c64f-fb37-4dda-b82a-3d44ccc3258f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:25.127625 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3e68ab-5d9d-4fbe-a201-735d55661469 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:25.160016 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:25.184727 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e229c64f-fb37-4dda-b82a-3d44ccc3258f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:25.185105 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789bffb8-f30e-461c-ae78-bbcf9809842c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:25.185735 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789bffb8-f30e-461c-ae78-bbcf9809842c service neutron] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:25.186040 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789bffb8-f30e-461c-ae78-bbcf9809842c service neutron] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:25.186323 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e229c64f-fb37-4dda-b82a-3d44ccc3258f service nova] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:25.186435 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789bffb8-f30e-461c-ae78-bbcf9809842c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:25.190152 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:25.201112 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 794/1587] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:55:25 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:55:25.204218 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e229c64f-fb37-4dda-b82a-3d44ccc3258f service nova] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:25.258406 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789bffb8-f30e-461c-ae78-bbcf9809842c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:25.260041 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789bffb8-f30e-461c-ae78-bbcf9809842c service neutron] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:25.270466 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 794/1588] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:55:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:55:25.271182 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789bffb8-f30e-461c-ae78-bbcf9809842c service neutron] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:25.274011 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e229c64f-fb37-4dda-b82a-3d44ccc3258f service nova] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:25.282072 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e899640-4aa2-4d4f-a754-f2f448adc02f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:25.282300 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789bffb8-f30e-461c-ae78-bbcf9809842c service neutron] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:25.282928 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e899640-4aa2-4d4f-a754-f2f448adc02f admin admin] REQUEST_METHOD: `PUT` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:25.283184 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e899640-4aa2-4d4f-a754-f2f448adc02f admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:25.283546 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e899640-4aa2-4d4f-a754-f2f448adc02f admin admin] PATH_INFO: `/v3/projects/3809ac47fe214dda92729d4c89648a17/users/993697d33b7841668b1eabcdcd6b40f9/roles/b674fd0a592e4e0cada811bba5ceea4d` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:25.287090 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735c6c45-3cd4-473c-851b-38fce846f69e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:25.287736 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735c6c45-3cd4-473c-851b-38fce846f69e admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:25.288052 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735c6c45-3cd4-473c-851b-38fce846f69e admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:25.293282 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735c6c45-3cd4-473c-851b-38fce846f69e admin admin] PATH_INFO: `/v3/projects` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:25.293282 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735c6c45-3cd4-473c-851b-38fce846f69e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:25.293282 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735c6c45-3cd4-473c-851b-38fce846f69e admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:25.303160 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e899640-4aa2-4d4f-a754-f2f448adc02f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3809ac47fe214dda92729d4c89648a17, user_id=993697d33b7841668b1eabcdcd6b40f9, role_id=b674fd0a592e4e0cada811bba5ceea4d)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:25.306031 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e899640-4aa2-4d4f-a754-f2f448adc02f admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:25.431493 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 795/1589] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 08:55:25 2022] PUT /identity/v3/projects/3809ac47fe214dda92729d4c89648a17/users/993697d33b7841668b1eabcdcd6b40f9/roles/b674fd0a592e4e0cada811bba5ceea4d => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:55:25.431493 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e899640-4aa2-4d4f-a754-f2f448adc02f admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:25.453262 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 795/1590] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 08:55:25 2022] POST /identity/v3/projects => generated 389 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:55:25.454457 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735c6c45-3cd4-473c-851b-38fce846f69e admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:25.526907 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e899640-4aa2-4d4f-a754-f2f448adc02f admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:25.544805 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735c6c45-3cd4-473c-851b-38fce846f69e admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:25.550907 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243a8f5d-3c73-4a1c-b9f1-448ed1b5b8cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:25.551538 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243a8f5d-3c73-4a1c-b9f1-448ed1b5b8cc admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:25.551841 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243a8f5d-3c73-4a1c-b9f1-448ed1b5b8cc admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:25.552134 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243a8f5d-3c73-4a1c-b9f1-448ed1b5b8cc admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:25.553388 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243a8f5d-3c73-4a1c-b9f1-448ed1b5b8cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:25.555371 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243a8f5d-3c73-4a1c-b9f1-448ed1b5b8cc admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:25.569327 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 796/1591] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:55:25 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:55:25.570224 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243a8f5d-3c73-4a1c-b9f1-448ed1b5b8cc admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:25.583135 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243a8f5d-3c73-4a1c-b9f1-448ed1b5b8cc admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:25.588937 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63454bc7-d87f-40ba-be52-a6e41bb7d192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:25.589718 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63454bc7-d87f-40ba-be52-a6e41bb7d192 admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:25.590000 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63454bc7-d87f-40ba-be52-a6e41bb7d192 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:25.590327 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63454bc7-d87f-40ba-be52-a6e41bb7d192 admin admin] PATH_INFO: `/v3/users` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:25.591153 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63454bc7-d87f-40ba-be52-a6e41bb7d192 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:25.593738 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63454bc7-d87f-40ba-be52-a6e41bb7d192 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:25.598790 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884428fd-acbe-446e-864d-c499e97141f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:25.599484 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884428fd-acbe-446e-864d-c499e97141f9 admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:25.599706 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884428fd-acbe-446e-864d-c499e97141f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:25.600024 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884428fd-acbe-446e-864d-c499e97141f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:25.600692 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884428fd-acbe-446e-864d-c499e97141f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:25.603068 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884428fd-acbe-446e-864d-c499e97141f9 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:25.682312 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 797/1592] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 08:55:25 2022] POST /identity/v3/users => generated 296 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:55:25.682312 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63454bc7-d87f-40ba-be52-a6e41bb7d192 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:25.704206 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63454bc7-d87f-40ba-be52-a6e41bb7d192 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:25.709863 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92056931-1059-4d6f-95b3-1729ec0ff349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:25.710695 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92056931-1059-4d6f-95b3-1729ec0ff349 admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:25.712801 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92056931-1059-4d6f-95b3-1729ec0ff349 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:25.712801 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92056931-1059-4d6f-95b3-1729ec0ff349 admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:25.712801 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92056931-1059-4d6f-95b3-1729ec0ff349 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:25.714442 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92056931-1059-4d6f-95b3-1729ec0ff349 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:25.730023 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 798/1593] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:55:25 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:55:25.730023 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce63fa0-2059-4038-912c-d619d2e100ce None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:25.730023 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce63fa0-2059-4038-912c-d619d2e100ce None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:25.730023 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce63fa0-2059-4038-912c-d619d2e100ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:25.740183 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 796/1594] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 08:55:25 2022] POST /identity/v3/projects => generated 377 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:55:25.742196 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884428fd-acbe-446e-864d-c499e97141f9 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:25.749934 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-9ce63fa0-2059-4038-912c-d619d2e100ce None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:25.751165 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-9ce63fa0-2059-4038-912c-d619d2e100ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:25.757563 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:25.772183 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:25.820258 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884428fd-acbe-446e-864d-c499e97141f9 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:25.825447 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d897f082-3e65-4b59-b4ea-73883e63eaa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:25.826168 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d897f082-3e65-4b59-b4ea-73883e63eaa2 admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:25.827323 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d897f082-3e65-4b59-b4ea-73883e63eaa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:25.827472 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 799/1595] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:55:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:25.827574 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d897f082-3e65-4b59-b4ea-73883e63eaa2 admin admin] PATH_INFO: `/v3/users` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:25.828061 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce63fa0-2059-4038-912c-d619d2e100ce None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:25.828273 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d897f082-3e65-4b59-b4ea-73883e63eaa2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:25.838482 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce63fa0-2059-4038-912c-d619d2e100ce None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:25.838933 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d897f082-3e65-4b59-b4ea-73883e63eaa2 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:25.843811 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062ea57c-9931-4740-a057-92ea86b67324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:25.844425 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062ea57c-9931-4740-a057-92ea86b67324 admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:25.844745 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062ea57c-9931-4740-a057-92ea86b67324 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:25.845008 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062ea57c-9931-4740-a057-92ea86b67324 admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:25.845656 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062ea57c-9931-4740-a057-92ea86b67324 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:25.847254 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062ea57c-9931-4740-a057-92ea86b67324 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:25.862858 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 800/1596] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:55:25 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:55:25.864201 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062ea57c-9931-4740-a057-92ea86b67324 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:25.874585 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062ea57c-9931-4740-a057-92ea86b67324 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:25.883758 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8c894a-30a9-497d-af0e-98ee743363c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:25.884392 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8c894a-30a9-497d-af0e-98ee743363c1 admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:25.884753 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8c894a-30a9-497d-af0e-98ee743363c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:25.884977 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8c894a-30a9-497d-af0e-98ee743363c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:25.885609 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8c894a-30a9-497d-af0e-98ee743363c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:25.887547 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8c894a-30a9-497d-af0e-98ee743363c1 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:25.900647 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 801/1597] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:55:25 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:55:25.906476 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bc5f39-cc9a-493c-91ed-44e4f4186845 None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:25.906476 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bc5f39-cc9a-493c-91ed-44e4f4186845 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:25.906476 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bc5f39-cc9a-493c-91ed-44e4f4186845 None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:25.909656 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 802/1598] 10.222.0.55 () {54 vars in 962 bytes} [Wed Aug 24 08:55:25 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:55:25.910992 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bc5f39-cc9a-493c-91ed-44e4f4186845 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:25.925866 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bc5f39-cc9a-493c-91ed-44e4f4186845 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:25.931420 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02312a3f-9a0f-4a90-b348-749f892a09a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:25.932064 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02312a3f-9a0f-4a90-b348-749f892a09a8 admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:25.932334 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02312a3f-9a0f-4a90-b348-749f892a09a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:25.935328 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02312a3f-9a0f-4a90-b348-749f892a09a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:25.935328 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02312a3f-9a0f-4a90-b348-749f892a09a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:25.935328 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02312a3f-9a0f-4a90-b348-749f892a09a8 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:25.945955 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 803/1599] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:55:25 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:55:25.946699 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02312a3f-9a0f-4a90-b348-749f892a09a8 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:25.956706 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02312a3f-9a0f-4a90-b348-749f892a09a8 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:25.961363 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23789ee-4e9f-4293-8fd2-8ccfa67d901a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:25.961989 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23789ee-4e9f-4293-8fd2-8ccfa67d901a admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:25.962260 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23789ee-4e9f-4293-8fd2-8ccfa67d901a admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:25.962530 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23789ee-4e9f-4293-8fd2-8ccfa67d901a admin admin] PATH_INFO: `/v3/users` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:25.963149 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23789ee-4e9f-4293-8fd2-8ccfa67d901a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:25.965096 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23789ee-4e9f-4293-8fd2-8ccfa67d901a admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:25.965656 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 797/1600] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 08:55:25 2022] POST /identity/v3/users => generated 295 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:55:25.966497 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d897f082-3e65-4b59-b4ea-73883e63eaa2 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:25.968250 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-d897f082-3e65-4b59-b4ea-73883e63eaa2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:26.030669 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d897f082-3e65-4b59-b4ea-73883e63eaa2 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:26.036642 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8af9cf-d118-4036-88b9-8ce7df02b99d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:26.036642 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8af9cf-d118-4036-88b9-8ce7df02b99d service neutron] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:26.036642 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8af9cf-d118-4036-88b9-8ce7df02b99d service neutron] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:26.036642 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8af9cf-d118-4036-88b9-8ce7df02b99d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:26.046105 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8af9cf-d118-4036-88b9-8ce7df02b99d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:26.047575 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8af9cf-d118-4036-88b9-8ce7df02b99d service neutron] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:26.059738 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 798/1601] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:55:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:55:26.060603 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8af9cf-d118-4036-88b9-8ce7df02b99d service neutron] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:26.071462 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8af9cf-d118-4036-88b9-8ce7df02b99d service neutron] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:26.077828 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c40018-14af-4d26-a080-09c143f01f2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:26.078694 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c40018-14af-4d26-a080-09c143f01f2d admin admin] REQUEST_METHOD: `PUT` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:26.101626 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c40018-14af-4d26-a080-09c143f01f2d admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:26.101626 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c40018-14af-4d26-a080-09c143f01f2d admin admin] PATH_INFO: `/v3/projects/2d8fe1b63acc4365ab4048772c6fccce/users/d32c786deac446a4bfb7973701ebd40f/roles/b674fd0a592e4e0cada811bba5ceea4d` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:26.101626 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c40018-14af-4d26-a080-09c143f01f2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d8fe1b63acc4365ab4048772c6fccce, user_id=d32c786deac446a4bfb7973701ebd40f, role_id=b674fd0a592e4e0cada811bba5ceea4d)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:26.101626 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c40018-14af-4d26-a080-09c143f01f2d admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:26.148315 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 804/1602] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 08:55:25 2022] POST /identity/v3/users => generated 289 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:55:26.149210 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23789ee-4e9f-4293-8fd2-8ccfa67d901a admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:26.160111 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23789ee-4e9f-4293-8fd2-8ccfa67d901a admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:26.165756 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f57293-db23-4009-b481-bb06fc347ded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:26.166377 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f57293-db23-4009-b481-bb06fc347ded admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:26.166734 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f57293-db23-4009-b481-bb06fc347ded admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:26.166944 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f57293-db23-4009-b481-bb06fc347ded admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:26.167587 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f57293-db23-4009-b481-bb06fc347ded admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:26.169417 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f57293-db23-4009-b481-bb06fc347ded admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:26.180487 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 805/1603] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:55:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:55:26.181794 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd4c2db-4175-4f20-949a-a597dabc225c None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:26.182076 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd4c2db-4175-4f20-949a-a597dabc225c None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:26.182351 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd4c2db-4175-4f20-949a-a597dabc225c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:26.203236 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-1cd4c2db-4175-4f20-949a-a597dabc225c None None] MFA Rules not processed for user `f17da357e555420799ee681bffbcc268`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:26.204485 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-1cd4c2db-4175-4f20-949a-a597dabc225c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:26.249935 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 799/1604] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 08:55:26 2022] PUT /identity/v3/projects/2d8fe1b63acc4365ab4048772c6fccce/users/d32c786deac446a4bfb7973701ebd40f/roles/b674fd0a592e4e0cada811bba5ceea4d => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:55:26.250609 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c40018-14af-4d26-a080-09c143f01f2d admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:26.285099 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 806/1605] 10.222.0.55 () {58 vars in 1059 bytes} [Wed Aug 24 08:55:26 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:26.286141 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd4c2db-4175-4f20-949a-a597dabc225c None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:26.300434 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd4c2db-4175-4f20-949a-a597dabc225c None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:26.345830 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c40018-14af-4d26-a080-09c143f01f2d admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:26.359341 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3514eb63-6e74-4ded-924a-f9cea5b24a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:26.359341 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3514eb63-6e74-4ded-924a-f9cea5b24a0f admin admin] REQUEST_METHOD: `PUT` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:26.359341 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3514eb63-6e74-4ded-924a-f9cea5b24a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:26.359341 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3514eb63-6e74-4ded-924a-f9cea5b24a0f admin admin] PATH_INFO: `/v3/projects/93c57966de00413ebea16616f7c9c25c/users/943e0762917a468dba61b9fd83d4f58e/roles/b674fd0a592e4e0cada811bba5ceea4d` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:26.371243 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3514eb63-6e74-4ded-924a-f9cea5b24a0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=93c57966de00413ebea16616f7c9c25c, user_id=943e0762917a468dba61b9fd83d4f58e, role_id=b674fd0a592e4e0cada811bba5ceea4d)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:26.372273 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:26.374139 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3514eb63-6e74-4ded-924a-f9cea5b24a0f admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:26.379289 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68d7498-4d8d-459d-b239-56413510fc02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:26.379916 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68d7498-4d8d-459d-b239-56413510fc02 admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:26.380320 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68d7498-4d8d-459d-b239-56413510fc02 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:26.380626 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68d7498-4d8d-459d-b239-56413510fc02 admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:26.381355 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68d7498-4d8d-459d-b239-56413510fc02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:26.382982 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68d7498-4d8d-459d-b239-56413510fc02 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:26.393500 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 807/1606] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:55:26 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:55:26.394431 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68d7498-4d8d-459d-b239-56413510fc02 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:26.405571 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68d7498-4d8d-459d-b239-56413510fc02 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:26.410214 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733dd178-a1ed-45c9-a8b6-023ff2516e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:26.410817 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733dd178-a1ed-45c9-a8b6-023ff2516e18 admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:26.411094 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733dd178-a1ed-45c9-a8b6-023ff2516e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:26.411364 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733dd178-a1ed-45c9-a8b6-023ff2516e18 admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:26.412010 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733dd178-a1ed-45c9-a8b6-023ff2516e18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:26.414232 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733dd178-a1ed-45c9-a8b6-023ff2516e18 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:26.424250 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 808/1607] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:55:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:55:26.428000 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733dd178-a1ed-45c9-a8b6-023ff2516e18 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:26.434864 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733dd178-a1ed-45c9-a8b6-023ff2516e18 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:26.439447 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6eec59-5396-4f5d-b3d4-cba1b70bb240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:26.440080 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6eec59-5396-4f5d-b3d4-cba1b70bb240 admin admin] REQUEST_METHOD: `PUT` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:26.440385 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6eec59-5396-4f5d-b3d4-cba1b70bb240 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:26.440685 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6eec59-5396-4f5d-b3d4-cba1b70bb240 admin admin] PATH_INFO: `/v3/projects/c966c27f100846ca8cf06122ddc8c408/users/f56524e822e641bc9bb11eddfe27f85d/roles/b674fd0a592e4e0cada811bba5ceea4d` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:26.458427 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6eec59-5396-4f5d-b3d4-cba1b70bb240 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c966c27f100846ca8cf06122ddc8c408, user_id=f56524e822e641bc9bb11eddfe27f85d, role_id=b674fd0a592e4e0cada811bba5ceea4d)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:26.463177 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6eec59-5396-4f5d-b3d4-cba1b70bb240 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:26.464987 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 800/1608] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 08:55:26 2022] PUT /identity/v3/projects/93c57966de00413ebea16616f7c9c25c/users/943e0762917a468dba61b9fd83d4f58e/roles/b674fd0a592e4e0cada811bba5ceea4d => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:55:26.476627 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081d5bf5-849e-4604-b024-7598db8de0a6 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:26.476627 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081d5bf5-849e-4604-b024-7598db8de0a6 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:26.476627 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081d5bf5-849e-4604-b024-7598db8de0a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:26.478588 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:26.489818 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-081d5bf5-849e-4604-b024-7598db8de0a6 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:26.491071 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-081d5bf5-849e-4604-b024-7598db8de0a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:26.563941 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 801/1609] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:55:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:26.563941 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8d95d8-55e8-4c40-81d6-2e68c3870726 None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:26.563941 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8d95d8-55e8-4c40-81d6-2e68c3870726 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:26.563941 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8d95d8-55e8-4c40-81d6-2e68c3870726 None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:26.570431 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 802/1610] 10.222.0.55 () {54 vars in 912 bytes} [Wed Aug 24 08:55:26 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:55:26.571436 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8d95d8-55e8-4c40-81d6-2e68c3870726 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:26.586658 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8d95d8-55e8-4c40-81d6-2e68c3870726 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:26.601785 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 809/1611] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 08:55:26 2022] PUT /identity/v3/projects/c966c27f100846ca8cf06122ddc8c408/users/f56524e822e641bc9bb11eddfe27f85d/roles/b674fd0a592e4e0cada811bba5ceea4d => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:55:26.603404 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6eec59-5396-4f5d-b3d4-cba1b70bb240 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:26.657343 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ddbd24-1f67-4de1-9c40-c203a2cb77e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:26.658685 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ddbd24-1f67-4de1-9c40-c203a2cb77e9 admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:26.659341 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ddbd24-1f67-4de1-9c40-c203a2cb77e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:26.659964 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ddbd24-1f67-4de1-9c40-c203a2cb77e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:26.661205 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ddbd24-1f67-4de1-9c40-c203a2cb77e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:26.663980 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ddbd24-1f67-4de1-9c40-c203a2cb77e9 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:26.682289 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 803/1612] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:55:26 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:55:26.684585 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7080eb5-7284-4c40-9eb3-1b74515eaf39 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:26.685154 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7080eb5-7284-4c40-9eb3-1b74515eaf39 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:26.685441 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7080eb5-7284-4c40-9eb3-1b74515eaf39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:26.688378 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6eec59-5396-4f5d-b3d4-cba1b70bb240 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:26.698607 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d807130-c51e-4748-bfff-de1768bcf72d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:26.699360 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d807130-c51e-4748-bfff-de1768bcf72d admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:26.699704 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d807130-c51e-4748-bfff-de1768bcf72d admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:26.700092 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d807130-c51e-4748-bfff-de1768bcf72d admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:26.700921 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d807130-c51e-4748-bfff-de1768bcf72d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:26.702639 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d807130-c51e-4748-bfff-de1768bcf72d admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:26.717072 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 810/1613] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:55:26 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:55:26.720565 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc080cbf-dc4b-4155-bcfa-bdd47f11372c None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:26.720874 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc080cbf-dc4b-4155-bcfa-bdd47f11372c None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:26.721169 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc080cbf-dc4b-4155-bcfa-bdd47f11372c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:26.732644 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-e7080eb5-7284-4c40-9eb3-1b74515eaf39 None None] MFA Rules not processed for user `943e0762917a468dba61b9fd83d4f58e`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:26.733956 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-e7080eb5-7284-4c40-9eb3-1b74515eaf39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:26.748061 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-fc080cbf-dc4b-4155-bcfa-bdd47f11372c None None] MFA Rules not processed for user `f17da357e555420799ee681bffbcc268`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:26.751079 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-fc080cbf-dc4b-4155-bcfa-bdd47f11372c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:26.842671 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 804/1614] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:55:26 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:26.844765 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7080eb5-7284-4c40-9eb3-1b74515eaf39 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:26.869632 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 811/1615] 10.222.0.55 () {58 vars in 1009 bytes} [Wed Aug 24 08:55:26 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:26.871528 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4133e09e-2c60-43cc-ac38-afdd00afe2de None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:26.872783 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4133e09e-2c60-43cc-ac38-afdd00afe2de None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:26.872783 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4133e09e-2c60-43cc-ac38-afdd00afe2de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:26.904055 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-4133e09e-2c60-43cc-ac38-afdd00afe2de None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:26.905405 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-4133e09e-2c60-43cc-ac38-afdd00afe2de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:26.917741 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:26.919565 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 812/1616] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:55:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:26.925687 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4133e09e-2c60-43cc-ac38-afdd00afe2de None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:26.947470 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4133e09e-2c60-43cc-ac38-afdd00afe2de None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:26.949142 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7080eb5-7284-4c40-9eb3-1b74515eaf39 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:26.954295 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc5e047-6a46-4b2c-a0e4-e487081d15df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:26.954957 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc5e047-6a46-4b2c-a0e4-e487081d15df service neutron] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:26.955304 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc5e047-6a46-4b2c-a0e4-e487081d15df service neutron] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:26.955676 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc5e047-6a46-4b2c-a0e4-e487081d15df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:26.958285 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16361cb2-34c4-407c-ba4b-76b04565306c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:26.959017 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16361cb2-34c4-407c-ba4b-76b04565306c admin admin] REQUEST_METHOD: `PUT` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:26.960073 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16361cb2-34c4-407c-ba4b-76b04565306c admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:26.960183 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16361cb2-34c4-407c-ba4b-76b04565306c admin admin] PATH_INFO: `/v3/projects/a69447ca2f314084925442b7f8c9ee2e/users/1105970982014891902ebbb9904a57a5/roles/b674fd0a592e4e0cada811bba5ceea4d` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:26.973401 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc5e047-6a46-4b2c-a0e4-e487081d15df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:26.973401 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc5e047-6a46-4b2c-a0e4-e487081d15df service neutron] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:26.985527 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 805/1617] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:55:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:55:26.986916 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc5e047-6a46-4b2c-a0e4-e487081d15df service neutron] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:26.996139 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16361cb2-34c4-407c-ba4b-76b04565306c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a69447ca2f314084925442b7f8c9ee2e, user_id=1105970982014891902ebbb9904a57a5, role_id=b674fd0a592e4e0cada811bba5ceea4d)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:26.999223 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16361cb2-34c4-407c-ba4b-76b04565306c admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:26.999412 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc5e047-6a46-4b2c-a0e4-e487081d15df service neutron] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:27.005518 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ae9b6f-0c1b-44ac-95e8-246604f15cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:27.006355 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ae9b6f-0c1b-44ac-95e8-246604f15cf4 admin admin] REQUEST_METHOD: `PUT` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:27.006701 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ae9b6f-0c1b-44ac-95e8-246604f15cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:27.007047 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ae9b6f-0c1b-44ac-95e8-246604f15cf4 admin admin] PATH_INFO: `/v3/projects/02d5c74d88a24eb9814f49a6ff4eca56/users/f5d7aee76519492e81b0048edebba36f/roles/b674fd0a592e4e0cada811bba5ceea4d` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:27.025919 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ae9b6f-0c1b-44ac-95e8-246604f15cf4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02d5c74d88a24eb9814f49a6ff4eca56, user_id=f5d7aee76519492e81b0048edebba36f, role_id=b674fd0a592e4e0cada811bba5ceea4d)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:27.033255 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ae9b6f-0c1b-44ac-95e8-246604f15cf4 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:27.101409 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 813/1618] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 08:55:26 2022] PUT /identity/v3/projects/a69447ca2f314084925442b7f8c9ee2e/users/1105970982014891902ebbb9904a57a5/roles/b674fd0a592e4e0cada811bba5ceea4d => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:55:27.102605 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16361cb2-34c4-407c-ba4b-76b04565306c admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:27.125910 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:27.180068 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16361cb2-34c4-407c-ba4b-76b04565306c admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:27.188020 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 806/1619] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 08:55:26 2022] PUT /identity/v3/projects/02d5c74d88a24eb9814f49a6ff4eca56/users/f5d7aee76519492e81b0048edebba36f/roles/b674fd0a592e4e0cada811bba5ceea4d => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:55:27.192577 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ae9b6f-0c1b-44ac-95e8-246604f15cf4 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:27.260994 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b341821f-7d58-49b8-87d2-110c6cd7ead5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:27.261646 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b341821f-7d58-49b8-87d2-110c6cd7ead5 admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:27.261931 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b341821f-7d58-49b8-87d2-110c6cd7ead5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:27.262378 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b341821f-7d58-49b8-87d2-110c6cd7ead5 admin admin] PATH_INFO: `/v3/projects` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:27.263020 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b341821f-7d58-49b8-87d2-110c6cd7ead5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:27.265010 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b341821f-7d58-49b8-87d2-110c6cd7ead5 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:27.265513 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ae9b6f-0c1b-44ac-95e8-246604f15cf4 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:27.269286 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306e758d-509a-4019-9a28-05891f3df7e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbb9ac667f04c35a63b2689df18efed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:27.272440 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306e758d-509a-4019-9a28-05891f3df7e3 service cinder] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:27.272440 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306e758d-509a-4019-9a28-05891f3df7e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:27.272440 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306e758d-509a-4019-9a28-05891f3df7e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:27.359352 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306e758d-509a-4019-9a28-05891f3df7e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:27.363954 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306e758d-509a-4019-9a28-05891f3df7e3 service cinder] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:27.377448 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 807/1620] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:55:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:55:27.378577 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306e758d-509a-4019-9a28-05891f3df7e3 service cinder] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:27.401837 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 814/1621] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 08:55:27 2022] POST /identity/v3/projects => generated 373 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:55:27.403184 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b0814d-2220-44b8-86f0-05e032264aa0 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:27.403461 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b0814d-2220-44b8-86f0-05e032264aa0 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:27.403984 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b0814d-2220-44b8-86f0-05e032264aa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:27.450316 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-32b0814d-2220-44b8-86f0-05e032264aa0 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:27.456485 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-32b0814d-2220-44b8-86f0-05e032264aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:27.496446 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306e758d-509a-4019-9a28-05891f3df7e3 service cinder] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:27.500941 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:27.572268 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed9ee90-da50-41cb-91a0-ea212d2db097 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:27.573869 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed9ee90-da50-41cb-91a0-ea212d2db097 service neutron] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:27.573869 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed9ee90-da50-41cb-91a0-ea212d2db097 service neutron] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:27.574073 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed9ee90-da50-41cb-91a0-ea212d2db097 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:27.576829 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 815/1622] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:55:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:27.578224 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e358fc-8657-488b-8fed-f78fd3451148 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:27.578516 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e358fc-8657-488b-8fed-f78fd3451148 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:27.585742 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e358fc-8657-488b-8fed-f78fd3451148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:27.601193 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed9ee90-da50-41cb-91a0-ea212d2db097 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:27.602836 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed9ee90-da50-41cb-91a0-ea212d2db097 service neutron] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:27.615747 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 808/1623] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:55:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:55:27.619776 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed9ee90-da50-41cb-91a0-ea212d2db097 service neutron] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:27.623853 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-18e358fc-8657-488b-8fed-f78fd3451148 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:27.626329 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-18e358fc-8657-488b-8fed-f78fd3451148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:27.628518 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed9ee90-da50-41cb-91a0-ea212d2db097 service neutron] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:27.643630 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ce76b7-f00e-473b-acd9-6d01a48b8b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:27.643630 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ce76b7-f00e-473b-acd9-6d01a48b8b24 admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:27.644089 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ce76b7-f00e-473b-acd9-6d01a48b8b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:27.644270 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ce76b7-f00e-473b-acd9-6d01a48b8b24 admin admin] PATH_INFO: `/v3/users` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:27.644912 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ce76b7-f00e-473b-acd9-6d01a48b8b24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:27.647851 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ce76b7-f00e-473b-acd9-6d01a48b8b24 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:27.659145 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 816/1624] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:55:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:27.660042 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e358fc-8657-488b-8fed-f78fd3451148 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:27.674629 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e358fc-8657-488b-8fed-f78fd3451148 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:27.678490 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3a17bf-e0f5-4f29-947c-88a70aaaa1fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:27.682965 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3a17bf-e0f5-4f29-947c-88a70aaaa1fe service neutron] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:27.682965 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3a17bf-e0f5-4f29-947c-88a70aaaa1fe service neutron] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:27.682965 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3a17bf-e0f5-4f29-947c-88a70aaaa1fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:27.691460 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3a17bf-e0f5-4f29-947c-88a70aaaa1fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:27.693228 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3a17bf-e0f5-4f29-947c-88a70aaaa1fe service neutron] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:27.705711 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 817/1625] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:55:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:55:27.707156 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49aea09-c615-4bbf-9846-7f09bb618e23 None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:27.711754 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49aea09-c615-4bbf-9846-7f09bb618e23 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:27.711754 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49aea09-c615-4bbf-9846-7f09bb618e23 None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:27.711754 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 818/1626] 10.222.0.55 () {54 vars in 912 bytes} [Wed Aug 24 08:55:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:55:27.711754 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49aea09-c615-4bbf-9846-7f09bb618e23 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:27.722495 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49aea09-c615-4bbf-9846-7f09bb618e23 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:27.726509 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c6c468-9cc3-4c3a-a1e2-d77db849ede9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:27.727262 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c6c468-9cc3-4c3a-a1e2-d77db849ede9 service neutron] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:27.727542 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c6c468-9cc3-4c3a-a1e2-d77db849ede9 service neutron] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:27.727809 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c6c468-9cc3-4c3a-a1e2-d77db849ede9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:27.738100 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 809/1627] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 08:55:27 2022] POST /identity/v3/users => generated 293 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:55:27.738864 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c6c468-9cc3-4c3a-a1e2-d77db849ede9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:27.740448 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a91723-8aa1-404f-b4ef-47304cb1d971 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:27.740448 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a91723-8aa1-404f-b4ef-47304cb1d971 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:27.740448 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a91723-8aa1-404f-b4ef-47304cb1d971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:27.740983 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c6c468-9cc3-4c3a-a1e2-d77db849ede9 service neutron] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:27.752396 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 819/1628] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:55:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:55:27.753524 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c6c468-9cc3-4c3a-a1e2-d77db849ede9 service neutron] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:27.769829 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c6c468-9cc3-4c3a-a1e2-d77db849ede9 service neutron] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:27.779575 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3c81e5-04f2-48c5-8b1b-26f28562467f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:27.780200 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3c81e5-04f2-48c5-8b1b-26f28562467f admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:27.780510 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3c81e5-04f2-48c5-8b1b-26f28562467f admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:27.780763 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3c81e5-04f2-48c5-8b1b-26f28562467f admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:27.781494 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3c81e5-04f2-48c5-8b1b-26f28562467f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:27.783107 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3c81e5-04f2-48c5-8b1b-26f28562467f admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:27.785565 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-69a91723-8aa1-404f-b4ef-47304cb1d971 None None] MFA Rules not processed for user `f17da357e555420799ee681bffbcc268`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:27.792929 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-69a91723-8aa1-404f-b4ef-47304cb1d971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:27.799892 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 820/1629] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:55:27 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:55:27.807791 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3c81e5-04f2-48c5-8b1b-26f28562467f admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:27.817320 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:27.832637 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3c81e5-04f2-48c5-8b1b-26f28562467f admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:27.855473 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcf9c20-f562-456f-a5e1-75a0c6738d95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:27.856408 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcf9c20-f562-456f-a5e1-75a0c6738d95 admin admin] REQUEST_METHOD: `PUT` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:27.856581 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcf9c20-f562-456f-a5e1-75a0c6738d95 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:27.857102 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcf9c20-f562-456f-a5e1-75a0c6738d95 admin admin] PATH_INFO: `/v3/projects/c469c437085945c4a7adb6e1aa148304/users/86011563534e4ba3b32f0639bed8c297/roles/b870cd1acaa744e1ae1d58b9ac283821` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:27.891882 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcf9c20-f562-456f-a5e1-75a0c6738d95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c469c437085945c4a7adb6e1aa148304, user_id=86011563534e4ba3b32f0639bed8c297, role_id=b870cd1acaa744e1ae1d58b9ac283821)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:27.894862 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcf9c20-f562-456f-a5e1-75a0c6738d95 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:28.155163 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 810/1630] 10.222.0.55 () {58 vars in 1009 bytes} [Wed Aug 24 08:55:27 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:28.155163 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a91723-8aa1-404f-b4ef-47304cb1d971 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:28.164366 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 821/1631] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 08:55:27 2022] PUT /identity/v3/projects/c469c437085945c4a7adb6e1aa148304/users/86011563534e4ba3b32f0639bed8c297/roles/b870cd1acaa744e1ae1d58b9ac283821 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:55:28.164366 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcf9c20-f562-456f-a5e1-75a0c6738d95 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:28.168975 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a91723-8aa1-404f-b4ef-47304cb1d971 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:28.224494 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:28.227713 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcf9c20-f562-456f-a5e1-75a0c6738d95 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:28.243066 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5501262f-7d4b-413c-b48f-830fdad6d89d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:28.244213 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5501262f-7d4b-413c-b48f-830fdad6d89d admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:28.244482 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5501262f-7d4b-413c-b48f-830fdad6d89d admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:28.245158 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5501262f-7d4b-413c-b48f-830fdad6d89d admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:28.246512 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5501262f-7d4b-413c-b48f-830fdad6d89d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:28.252105 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5501262f-7d4b-413c-b48f-830fdad6d89d admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:28.254227 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f69626-2909-499e-b037-1691757f9448 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbb9ac667f04c35a63b2689df18efed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:28.257990 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f69626-2909-499e-b037-1691757f9448 service cinder] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:28.258305 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f69626-2909-499e-b037-1691757f9448 service cinder] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:28.258572 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f69626-2909-499e-b037-1691757f9448 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:28.298165 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 822/1632] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:55:28 2022] GET /identity/v3/roles => generated 1420 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 08:55:28.299795 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5501262f-7d4b-413c-b48f-830fdad6d89d admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:28.335936 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5501262f-7d4b-413c-b48f-830fdad6d89d admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:28.349210 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fca62e-2d5e-48bb-9968-fa6441d9b11e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:28.350580 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fca62e-2d5e-48bb-9968-fa6441d9b11e admin admin] REQUEST_METHOD: `PUT` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:28.351058 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fca62e-2d5e-48bb-9968-fa6441d9b11e admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:28.351518 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fca62e-2d5e-48bb-9968-fa6441d9b11e admin admin] PATH_INFO: `/v3/projects/c469c437085945c4a7adb6e1aa148304/users/86011563534e4ba3b32f0639bed8c297/roles/b870cd1acaa744e1ae1d58b9ac283821` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:28.354633 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fca62e-2d5e-48bb-9968-fa6441d9b11e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c469c437085945c4a7adb6e1aa148304, user_id=86011563534e4ba3b32f0639bed8c297, role_id=b870cd1acaa744e1ae1d58b9ac283821)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:28.361389 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fca62e-2d5e-48bb-9968-fa6441d9b11e admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:28.401250 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 823/1633] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 08:55:28 2022] PUT /identity/v3/projects/c469c437085945c4a7adb6e1aa148304/users/86011563534e4ba3b32f0639bed8c297/roles/b870cd1acaa744e1ae1d58b9ac283821 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:55:28.413796 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b2749a-9618-43fb-aad6-8075a2b0194c None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:28.414397 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b2749a-9618-43fb-aad6-8075a2b0194c None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:28.415522 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b2749a-9618-43fb-aad6-8075a2b0194c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:28.439459 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f69626-2909-499e-b037-1691757f9448 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:28.441460 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f69626-2909-499e-b037-1691757f9448 service cinder] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:28.458686 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-71b2749a-9618-43fb-aad6-8075a2b0194c None None] MFA Rules not processed for user `86011563534e4ba3b32f0639bed8c297`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:28.460924 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-71b2749a-9618-43fb-aad6-8075a2b0194c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:28.473620 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 811/1634] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:55:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:55:28.571943 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 824/1635] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:55:28 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:55:28.581377 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f69626-2909-499e-b037-1691757f9448 service cinder] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:28.596699 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f69626-2909-499e-b037-1691757f9448 service cinder] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:28.629031 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b2749a-9618-43fb-aad6-8075a2b0194c None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:28.683172 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c8fbad-48a6-4314-98c7-be58b29f5362 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbb9ac667f04c35a63b2689df18efed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:28.684350 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c8fbad-48a6-4314-98c7-be58b29f5362 service cinder] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:28.685003 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c8fbad-48a6-4314-98c7-be58b29f5362 service cinder] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:28.685568 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c8fbad-48a6-4314-98c7-be58b29f5362 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:28.701658 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c8fbad-48a6-4314-98c7-be58b29f5362 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:28.703539 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c8fbad-48a6-4314-98c7-be58b29f5362 service cinder] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:28.714016 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b2749a-9618-43fb-aad6-8075a2b0194c None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:28.722850 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b643f9d-61ec-462c-a562-31e3597c57dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:28.723509 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b643f9d-61ec-462c-a562-31e3597c57dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:28.723791 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b643f9d-61ec-462c-a562-31e3597c57dc admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:28.724063 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b643f9d-61ec-462c-a562-31e3597c57dc admin admin] PATH_INFO: `/v3/users/5e8fd689c9af47ffac97e81255fc04f9` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:28.725382 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b643f9d-61ec-462c-a562-31e3597c57dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e8fd689c9af47ffac97e81255fc04f9)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:28.727573 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b643f9d-61ec-462c-a562-31e3597c57dc admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:28.733460 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 812/1636] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:55:28 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:55:28.768521 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:28.983889 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-3b643f9d-61ec-462c-a562-31e3597c57dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e8fd689c9af47ffac97e81255fc04f9'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:28.984711 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-3b643f9d-61ec-462c-a562-31e3597c57dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e8fd689c9af47ffac97e81255fc04f9'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:28.985406 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-3b643f9d-61ec-462c-a562-31e3597c57dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e8fd689c9af47ffac97e81255fc04f9'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:29.010448 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-3b643f9d-61ec-462c-a562-31e3597c57dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e8fd689c9af47ffac97e81255fc04f9'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:29.126403 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-3b643f9d-61ec-462c-a562-31e3597c57dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e8fd689c9af47ffac97e81255fc04f9'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:29.155546 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 825/1637] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 08:55:28 2022] DELETE /identity/v3/users/5e8fd689c9af47ffac97e81255fc04f9 => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:55:29.161768 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f812cf61-5340-49be-89e5-4fb7e3d4958f None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:29.162796 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f812cf61-5340-49be-89e5-4fb7e3d4958f None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:29.163313 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f812cf61-5340-49be-89e5-4fb7e3d4958f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:29.195192 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-f812cf61-5340-49be-89e5-4fb7e3d4958f None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:29.197271 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-f812cf61-5340-49be-89e5-4fb7e3d4958f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:29.346986 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 813/1638] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:55:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:29.384725 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b643f9d-61ec-462c-a562-31e3597c57dc admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:29.386455 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-3b643f9d-61ec-462c-a562-31e3597c57dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:29.521548 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b643f9d-61ec-462c-a562-31e3597c57dc admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:29.530430 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f092dc5c-7511-4a05-8c8a-44cfb8c1d1b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:29.531015 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f092dc5c-7511-4a05-8c8a-44cfb8c1d1b2 service neutron] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:29.531355 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f092dc5c-7511-4a05-8c8a-44cfb8c1d1b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:29.531634 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f092dc5c-7511-4a05-8c8a-44cfb8c1d1b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:29.562861 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f092dc5c-7511-4a05-8c8a-44cfb8c1d1b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:29.567030 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f092dc5c-7511-4a05-8c8a-44cfb8c1d1b2 service neutron] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:29.621374 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 826/1639] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:55:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:55:30.256940 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f812cf61-5340-49be-89e5-4fb7e3d4958f None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:30.261430 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-f812cf61-5340-49be-89e5-4fb7e3d4958f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:30.284211 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f812cf61-5340-49be-89e5-4fb7e3d4958f None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:30.303921 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48c7b0f-e916-428b-bfec-431c9443dbfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:30.303921 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48c7b0f-e916-428b-bfec-431c9443dbfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:30.303921 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48c7b0f-e916-428b-bfec-431c9443dbfc admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:30.303921 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48c7b0f-e916-428b-bfec-431c9443dbfc admin admin] PATH_INFO: `/v3/users/943e0762917a468dba61b9fd83d4f58e` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:30.304248 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48c7b0f-e916-428b-bfec-431c9443dbfc admin admin] RBAC: Authorizing `identity:delete_user(user_id=943e0762917a468dba61b9fd83d4f58e)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:30.304248 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: WARNING py.warnings [None req-e48c7b0f-e916-428b-bfec-431c9443dbfc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 08:55:30.304248 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: warnings.warn(msg) Aug 24 08:55:30.304248 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: Aug 24 08:55:30.307879 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48c7b0f-e916-428b-bfec-431c9443dbfc admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:30.383057 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:30.465813 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615b0d59-f7c6-4dc1-a822-bcbc4a66bd16 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:30.466384 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615b0d59-f7c6-4dc1-a822-bcbc4a66bd16 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:30.466384 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615b0d59-f7c6-4dc1-a822-bcbc4a66bd16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:30.521564 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-615b0d59-f7c6-4dc1-a822-bcbc4a66bd16 None None] MFA Rules not processed for user `d32c786deac446a4bfb7973701ebd40f`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:30.522822 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-615b0d59-f7c6-4dc1-a822-bcbc4a66bd16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:30.651748 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-e48c7b0f-e916-428b-bfec-431c9443dbfc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '943e0762917a468dba61b9fd83d4f58e'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:30.655208 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-e48c7b0f-e916-428b-bfec-431c9443dbfc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '943e0762917a468dba61b9fd83d4f58e'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:30.655208 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-e48c7b0f-e916-428b-bfec-431c9443dbfc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '943e0762917a468dba61b9fd83d4f58e'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:30.676189 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:30.699784 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-e48c7b0f-e916-428b-bfec-431c9443dbfc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '943e0762917a468dba61b9fd83d4f58e'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:30.700631 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 827/1640] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:55:30 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:55:30.724326 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615b0d59-f7c6-4dc1-a822-bcbc4a66bd16 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:30.730064 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-615b0d59-f7c6-4dc1-a822-bcbc4a66bd16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:30.771297 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615b0d59-f7c6-4dc1-a822-bcbc4a66bd16 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:30.779186 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca126c79-ef19-47c8-9762-a1f6ce9c8899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:30.780471 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca126c79-ef19-47c8-9762-a1f6ce9c8899 admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:30.780855 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca126c79-ef19-47c8-9762-a1f6ce9c8899 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:30.781180 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca126c79-ef19-47c8-9762-a1f6ce9c8899 admin admin] PATH_INFO: `/v3/projects` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:30.781989 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca126c79-ef19-47c8-9762-a1f6ce9c8899 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:30.784101 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca126c79-ef19-47c8-9762-a1f6ce9c8899 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:30.851322 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-e48c7b0f-e916-428b-bfec-431c9443dbfc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '943e0762917a468dba61b9fd83d4f58e'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:30.880060 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 814/1641] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 08:55:30 2022] DELETE /identity/v3/users/943e0762917a468dba61b9fd83d4f58e => generated 0 bytes in 624 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:55:30.886197 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cbb44a-4484-4494-a471-29f24e2ba01c None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:30.886510 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cbb44a-4484-4494-a471-29f24e2ba01c None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:30.886809 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cbb44a-4484-4494-a471-29f24e2ba01c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:30.947679 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-90cbb44a-4484-4494-a471-29f24e2ba01c None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:30.949191 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-90cbb44a-4484-4494-a471-29f24e2ba01c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:30.969551 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 828/1642] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 08:55:30 2022] POST /identity/v3/projects => generated 403 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:55:30.973708 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca126c79-ef19-47c8-9762-a1f6ce9c8899 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:31.094211 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 815/1643] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:55:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:31.094808 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca126c79-ef19-47c8-9762-a1f6ce9c8899 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:31.100831 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3ac06f-57a9-469a-85ef-60bcfde84565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:31.101422 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3ac06f-57a9-469a-85ef-60bcfde84565 admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:31.101703 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3ac06f-57a9-469a-85ef-60bcfde84565 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:31.101978 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3ac06f-57a9-469a-85ef-60bcfde84565 admin admin] PATH_INFO: `/v3/users` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:31.102611 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3ac06f-57a9-469a-85ef-60bcfde84565 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:31.104605 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3ac06f-57a9-469a-85ef-60bcfde84565 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:31.130893 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cbb44a-4484-4494-a471-29f24e2ba01c None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:31.135814 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-90cbb44a-4484-4494-a471-29f24e2ba01c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:31.262047 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cbb44a-4484-4494-a471-29f24e2ba01c None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:31.322554 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64a111d-f2bb-4cb1-9440-b5231a25a3fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:31.323567 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64a111d-f2bb-4cb1-9440-b5231a25a3fe service neutron] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:31.324040 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64a111d-f2bb-4cb1-9440-b5231a25a3fe service neutron] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:31.324648 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64a111d-f2bb-4cb1-9440-b5231a25a3fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:31.336433 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64a111d-f2bb-4cb1-9440-b5231a25a3fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:31.338255 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64a111d-f2bb-4cb1-9440-b5231a25a3fe service neutron] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:31.362098 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 829/1644] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 08:55:30 2022] POST /identity/v3/users => generated 308 bytes in 389 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:55:31.364334 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3ac06f-57a9-469a-85ef-60bcfde84565 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:31.365112 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-ea3ac06f-57a9-469a-85ef-60bcfde84565 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:31.367659 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 816/1645] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:55:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:55:31.368852 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64a111d-f2bb-4cb1-9440-b5231a25a3fe service neutron] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:31.386538 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64a111d-f2bb-4cb1-9440-b5231a25a3fe service neutron] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:31.392381 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3ac06f-57a9-469a-85ef-60bcfde84565 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:31.399455 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d655af5f-8137-4847-9dc3-3b467dab6de3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:31.400410 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d655af5f-8137-4847-9dc3-3b467dab6de3 admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:31.400904 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d655af5f-8137-4847-9dc3-3b467dab6de3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:31.401435 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d655af5f-8137-4847-9dc3-3b467dab6de3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:31.403058 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d655af5f-8137-4847-9dc3-3b467dab6de3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:31.403963 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4bcab5-0887-4452-ad57-c3a196beeb52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:31.404755 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4bcab5-0887-4452-ad57-c3a196beeb52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:31.405800 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d655af5f-8137-4847-9dc3-3b467dab6de3 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:31.406206 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4bcab5-0887-4452-ad57-c3a196beeb52 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:31.406608 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4bcab5-0887-4452-ad57-c3a196beeb52 admin admin] PATH_INFO: `/v3/projects/460e817ed5584689a98e649af18c4451` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:31.411180 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4bcab5-0887-4452-ad57-c3a196beeb52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=460e817ed5584689a98e649af18c4451)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:31.414293 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: WARNING py.warnings [None req-9c4bcab5-0887-4452-ad57-c3a196beeb52 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 08:55:31.414293 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: warnings.warn(msg) Aug 24 08:55:31.414293 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: Aug 24 08:55:31.415634 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4bcab5-0887-4452-ad57-c3a196beeb52 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:31.425737 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 817/1646] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:55:31 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:55:31.430190 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d655af5f-8137-4847-9dc3-3b467dab6de3 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:31.445753 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d655af5f-8137-4847-9dc3-3b467dab6de3 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:31.453714 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f63c98-b83f-480f-a32d-091c5c417dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:31.455844 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f63c98-b83f-480f-a32d-091c5c417dc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:31.456150 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f63c98-b83f-480f-a32d-091c5c417dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:31.456773 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f63c98-b83f-480f-a32d-091c5c417dc0 admin admin] PATH_INFO: `/v3/projects/a353c4943d18429a94aa9d93b67f04db/users/f2b2a9bcb9884fadaf78086b0cc4ea6d/roles/b870cd1acaa744e1ae1d58b9ac283821` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:31.487601 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f63c98-b83f-480f-a32d-091c5c417dc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a353c4943d18429a94aa9d93b67f04db, user_id=f2b2a9bcb9884fadaf78086b0cc4ea6d, role_id=b870cd1acaa744e1ae1d58b9ac283821)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:31.493288 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f63c98-b83f-480f-a32d-091c5c417dc0 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:31.629437 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 818/1647] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 08:55:31 2022] PUT /identity/v3/projects/a353c4943d18429a94aa9d93b67f04db/users/f2b2a9bcb9884fadaf78086b0cc4ea6d/roles/b870cd1acaa744e1ae1d58b9ac283821 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:55:31.633929 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f63c98-b83f-480f-a32d-091c5c417dc0 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:31.673290 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-9c4bcab5-0887-4452-ad57-c3a196beeb52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '460e817ed5584689a98e649af18c4451', 'request_id': 'req-9c4bcab5-0887-4452-ad57-c3a196beeb52'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:31.691618 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-9c4bcab5-0887-4452-ad57-c3a196beeb52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '460e817ed5584689a98e649af18c4451', 'request_id': 'req-9c4bcab5-0887-4452-ad57-c3a196beeb52'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:31.746663 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f63c98-b83f-480f-a32d-091c5c417dc0 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:31.768260 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-9c4bcab5-0887-4452-ad57-c3a196beeb52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '460e817ed5584689a98e649af18c4451', 'request_id': 'req-9c4bcab5-0887-4452-ad57-c3a196beeb52'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:31.809782 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-9c4bcab5-0887-4452-ad57-c3a196beeb52 admin admin] The token cache is being invalidate because project 460e817ed5584689a98e649af18c4451 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:55:31.810078 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-9c4bcab5-0887-4452-ad57-c3a196beeb52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:31.812399 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-9c4bcab5-0887-4452-ad57-c3a196beeb52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:31.817493 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 830/1648] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 08:55:31 2022] DELETE /identity/v3/projects/460e817ed5584689a98e649af18c4451 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:55:31.833118 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe91b030-bbe0-462d-8b97-979475217c82 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:31.833412 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe91b030-bbe0-462d-8b97-979475217c82 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:31.833690 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe91b030-bbe0-462d-8b97-979475217c82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:31.900033 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddf752f-9feb-4502-8304-5d7abeaa17bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:31.901258 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddf752f-9feb-4502-8304-5d7abeaa17bf admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:31.902087 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddf752f-9feb-4502-8304-5d7abeaa17bf admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:31.902570 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddf752f-9feb-4502-8304-5d7abeaa17bf admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:31.915189 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddf752f-9feb-4502-8304-5d7abeaa17bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:31.918320 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddf752f-9feb-4502-8304-5d7abeaa17bf admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:31.920467 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-fe91b030-bbe0-462d-8b97-979475217c82 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:31.921655 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:31.921946 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-fe91b030-bbe0-462d-8b97-979475217c82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:31.937792 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 819/1649] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:55:31 2022] GET /identity/v3/roles => generated 1420 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:55:31.941183 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddf752f-9feb-4502-8304-5d7abeaa17bf admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:31.944264 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-6ddf752f-9feb-4502-8304-5d7abeaa17bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:31.948066 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 831/1650] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:55:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:31.959917 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e03783e-546e-4200-885c-84efb8fa099a None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:31.960207 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e03783e-546e-4200-885c-84efb8fa099a None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:31.961358 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e03783e-546e-4200-885c-84efb8fa099a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:31.967027 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddf752f-9feb-4502-8304-5d7abeaa17bf admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:31.976761 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a944c41f-e258-4ba2-bc61-d6c7f6ea084a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:31.977865 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a944c41f-e258-4ba2-bc61-d6c7f6ea084a admin admin] REQUEST_METHOD: `PUT` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:31.978351 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a944c41f-e258-4ba2-bc61-d6c7f6ea084a admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:31.978801 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a944c41f-e258-4ba2-bc61-d6c7f6ea084a admin admin] PATH_INFO: `/v3/projects/a353c4943d18429a94aa9d93b67f04db/users/f2b2a9bcb9884fadaf78086b0cc4ea6d/roles/b870cd1acaa744e1ae1d58b9ac283821` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:31.982431 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a944c41f-e258-4ba2-bc61-d6c7f6ea084a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a353c4943d18429a94aa9d93b67f04db, user_id=f2b2a9bcb9884fadaf78086b0cc4ea6d, role_id=b870cd1acaa744e1ae1d58b9ac283821)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:31.986036 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a944c41f-e258-4ba2-bc61-d6c7f6ea084a admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:32.001273 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-8e03783e-546e-4200-885c-84efb8fa099a None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:32.003154 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-8e03783e-546e-4200-885c-84efb8fa099a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:32.017529 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 820/1651] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 08:55:31 2022] PUT /identity/v3/projects/a353c4943d18429a94aa9d93b67f04db/users/f2b2a9bcb9884fadaf78086b0cc4ea6d/roles/b870cd1acaa744e1ae1d58b9ac283821 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:55:32.100886 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a944c41f-e258-4ba2-bc61-d6c7f6ea084a admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:32.102861 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 832/1652] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:55:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:32.113097 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a944c41f-e258-4ba2-bc61-d6c7f6ea084a admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:32.122140 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb101e7-6756-4573-a093-ef2b766736b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:32.122599 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb101e7-6756-4573-a093-ef2b766736b5 admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:32.122866 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb101e7-6756-4573-a093-ef2b766736b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:32.123135 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb101e7-6756-4573-a093-ef2b766736b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:32.123928 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb101e7-6756-4573-a093-ef2b766736b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:55:32.124297 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb101e7-6756-4573-a093-ef2b766736b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:32.125972 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb101e7-6756-4573-a093-ef2b766736b5 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:32.155458 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e03783e-546e-4200-885c-84efb8fa099a None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:32.166343 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e03783e-546e-4200-885c-84efb8fa099a None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:32.166529 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 821/1653] 10.222.0.55 () {58 vars in 1153 bytes} [Wed Aug 24 08:55:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:55:32.171963 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eae39f-062d-46e4-b31f-67abb4050e5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:32.173221 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eae39f-062d-46e4-b31f-67abb4050e5f admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:32.173510 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eae39f-062d-46e4-b31f-67abb4050e5f admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:32.173784 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eae39f-062d-46e4-b31f-67abb4050e5f admin admin] PATH_INFO: `/v3/projects` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:32.174580 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26eae39f-062d-46e4-b31f-67abb4050e5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:32.176756 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26eae39f-062d-46e4-b31f-67abb4050e5f admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:32.359908 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 833/1654] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 08:55:32 2022] POST /identity/v3/projects => generated 375 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:55:32.361446 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb101e7-6756-4573-a093-ef2b766736b5 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:32.377345 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb101e7-6756-4573-a093-ef2b766736b5 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:32.492868 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eae39f-062d-46e4-b31f-67abb4050e5f admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:32.494778 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-26eae39f-062d-46e4-b31f-67abb4050e5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:32.550013 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78477b59-e5dd-4049-9dd9-d38f71588e26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:32.550977 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78477b59-e5dd-4049-9dd9-d38f71588e26 admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:32.550977 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78477b59-e5dd-4049-9dd9-d38f71588e26 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:32.552520 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78477b59-e5dd-4049-9dd9-d38f71588e26 admin admin] PATH_INFO: `/v3/users` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:32.553900 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78477b59-e5dd-4049-9dd9-d38f71588e26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:32.556062 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78477b59-e5dd-4049-9dd9-d38f71588e26 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:32.637615 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eae39f-062d-46e4-b31f-67abb4050e5f admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:32.656865 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f095a23-ad37-46fe-8e55-62f3aebeff19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:32.657593 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f095a23-ad37-46fe-8e55-62f3aebeff19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:32.662607 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f095a23-ad37-46fe-8e55-62f3aebeff19 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:32.662607 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f095a23-ad37-46fe-8e55-62f3aebeff19 admin admin] PATH_INFO: `/v3/projects/93c57966de00413ebea16616f7c9c25c` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:32.663048 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 822/1655] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 08:55:32 2022] POST /identity/v3/users => generated 288 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:55:32.663217 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f095a23-ad37-46fe-8e55-62f3aebeff19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93c57966de00413ebea16616f7c9c25c)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:32.663357 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78477b59-e5dd-4049-9dd9-d38f71588e26 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:32.664917 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f095a23-ad37-46fe-8e55-62f3aebeff19 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:32.688713 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78477b59-e5dd-4049-9dd9-d38f71588e26 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:32.708044 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef4f219-a527-44d1-be56-7be2646d4378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:32.710154 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef4f219-a527-44d1-be56-7be2646d4378 admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:32.710875 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef4f219-a527-44d1-be56-7be2646d4378 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:32.710875 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef4f219-a527-44d1-be56-7be2646d4378 admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:32.733676 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef4f219-a527-44d1-be56-7be2646d4378 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:32.739143 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef4f219-a527-44d1-be56-7be2646d4378 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:32.782515 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 823/1656] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:55:32 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 08:55:32.783080 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef4f219-a527-44d1-be56-7be2646d4378 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:32.811874 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef4f219-a527-44d1-be56-7be2646d4378 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:32.829514 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ffa00b-51f8-4d19-8045-20bad598209d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:32.830149 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ffa00b-51f8-4d19-8045-20bad598209d admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:32.830424 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ffa00b-51f8-4d19-8045-20bad598209d admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:32.830694 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ffa00b-51f8-4d19-8045-20bad598209d admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:32.831342 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ffa00b-51f8-4d19-8045-20bad598209d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:32.837490 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ffa00b-51f8-4d19-8045-20bad598209d admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:32.884303 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 824/1657] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:55:32 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:55:32.884303 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ffa00b-51f8-4d19-8045-20bad598209d admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:32.938208 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ffa00b-51f8-4d19-8045-20bad598209d admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:32.938589 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-0f095a23-ad37-46fe-8e55-62f3aebeff19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93c57966de00413ebea16616f7c9c25c', 'request_id': 'req-0f095a23-ad37-46fe-8e55-62f3aebeff19'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:32.938589 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-0f095a23-ad37-46fe-8e55-62f3aebeff19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93c57966de00413ebea16616f7c9c25c', 'request_id': 'req-0f095a23-ad37-46fe-8e55-62f3aebeff19'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:33.004944 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bb3ec3-1b33-43b5-9b63-c2b9daf5b383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:33.005808 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bb3ec3-1b33-43b5-9b63-c2b9daf5b383 admin admin] REQUEST_METHOD: `PUT` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:33.006281 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bb3ec3-1b33-43b5-9b63-c2b9daf5b383 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:33.006672 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bb3ec3-1b33-43b5-9b63-c2b9daf5b383 admin admin] PATH_INFO: `/v3/projects/d7650312b93a44ceb07ebd745fd83f05/users/78df990d90fe43e0bc3a0fd1cd834715/roles/b674fd0a592e4e0cada811bba5ceea4d` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:33.035972 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bb3ec3-1b33-43b5-9b63-c2b9daf5b383 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7650312b93a44ceb07ebd745fd83f05, user_id=78df990d90fe43e0bc3a0fd1cd834715, role_id=b674fd0a592e4e0cada811bba5ceea4d)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:33.041063 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bb3ec3-1b33-43b5-9b63-c2b9daf5b383 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:33.109524 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-0f095a23-ad37-46fe-8e55-62f3aebeff19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93c57966de00413ebea16616f7c9c25c', 'request_id': 'req-0f095a23-ad37-46fe-8e55-62f3aebeff19'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:33.132586 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-0f095a23-ad37-46fe-8e55-62f3aebeff19 admin admin] The token cache is being invalidate because project 93c57966de00413ebea16616f7c9c25c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:55:33.133196 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-0f095a23-ad37-46fe-8e55-62f3aebeff19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:33.134049 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-0f095a23-ad37-46fe-8e55-62f3aebeff19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:33.141248 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 834/1658] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 08:55:32 2022] DELETE /identity/v3/projects/93c57966de00413ebea16616f7c9c25c => generated 0 bytes in 645 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:55:33.142542 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f095a23-ad37-46fe-8e55-62f3aebeff19 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:33.144878 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-0f095a23-ad37-46fe-8e55-62f3aebeff19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:33.162296 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f095a23-ad37-46fe-8e55-62f3aebeff19 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:33.169109 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2c99f8-d385-4d61-acec-adf81b28d24a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:33.181643 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2c99f8-d385-4d61-acec-adf81b28d24a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:33.181643 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2c99f8-d385-4d61-acec-adf81b28d24a admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:33.181643 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2c99f8-d385-4d61-acec-adf81b28d24a admin admin] PATH_INFO: `/v3/users/86011563534e4ba3b32f0639bed8c297` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:33.181643 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2c99f8-d385-4d61-acec-adf81b28d24a admin admin] RBAC: Authorizing `identity:delete_user(user_id=86011563534e4ba3b32f0639bed8c297)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:33.181643 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2c99f8-d385-4d61-acec-adf81b28d24a admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:33.253695 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 825/1659] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 08:55:32 2022] PUT /identity/v3/projects/d7650312b93a44ceb07ebd745fd83f05/users/78df990d90fe43e0bc3a0fd1cd834715/roles/b674fd0a592e4e0cada811bba5ceea4d => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:55:33.264585 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aeca9d-1488-4e5b-b948-851aa364a6f0 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:33.265201 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aeca9d-1488-4e5b-b948-851aa364a6f0 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:33.265628 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aeca9d-1488-4e5b-b948-851aa364a6f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:33.333475 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-f7aeca9d-1488-4e5b-b948-851aa364a6f0 None None] MFA Rules not processed for user `78df990d90fe43e0bc3a0fd1cd834715`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:33.335649 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-f7aeca9d-1488-4e5b-b948-851aa364a6f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:33.431285 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 826/1660] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:55:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:55:33.436078 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aeca9d-1488-4e5b-b948-851aa364a6f0 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:33.442670 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-f7aeca9d-1488-4e5b-b948-851aa364a6f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:33.496464 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-1e2c99f8-d385-4d61-acec-adf81b28d24a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86011563534e4ba3b32f0639bed8c297'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:33.499131 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-1e2c99f8-d385-4d61-acec-adf81b28d24a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86011563534e4ba3b32f0639bed8c297'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:33.500982 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-1e2c99f8-d385-4d61-acec-adf81b28d24a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86011563534e4ba3b32f0639bed8c297'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:33.544680 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:33.561474 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-1e2c99f8-d385-4d61-acec-adf81b28d24a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86011563534e4ba3b32f0639bed8c297'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:33.604182 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aeca9d-1488-4e5b-b948-851aa364a6f0 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:33.609921 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a9bc3b-dc9b-4b66-b6f1-1fc1101577d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:33.611344 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a9bc3b-dc9b-4b66-b6f1-1fc1101577d3 admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:33.612485 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a9bc3b-dc9b-4b66-b6f1-1fc1101577d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:33.612962 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a9bc3b-dc9b-4b66-b6f1-1fc1101577d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:33.614187 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a9bc3b-dc9b-4b66-b6f1-1fc1101577d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:33.617414 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a9bc3b-dc9b-4b66-b6f1-1fc1101577d3 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:33.645076 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-1e2c99f8-d385-4d61-acec-adf81b28d24a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86011563534e4ba3b32f0639bed8c297'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:33.663278 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 835/1661] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 08:55:33 2022] DELETE /identity/v3/users/86011563534e4ba3b32f0639bed8c297 => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:55:33.707405 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 827/1662] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 08:55:33 2022] POST /identity/v3/projects => generated 377 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:55:33.712204 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2c99f8-d385-4d61-acec-adf81b28d24a admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:33.713815 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-1e2c99f8-d385-4d61-acec-adf81b28d24a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:33.867600 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2c99f8-d385-4d61-acec-adf81b28d24a admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:33.988001 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ef135f-50b2-4f00-b48d-5660f8a61090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:33.989805 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ef135f-50b2-4f00-b48d-5660f8a61090 admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:33.990775 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ef135f-50b2-4f00-b48d-5660f8a61090 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:33.991184 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ef135f-50b2-4f00-b48d-5660f8a61090 admin admin] PATH_INFO: `/v3/users` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:33.992048 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ef135f-50b2-4f00-b48d-5660f8a61090 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:33.994996 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ef135f-50b2-4f00-b48d-5660f8a61090 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:34.138843 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 836/1663] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 08:55:33 2022] POST /identity/v3/users => generated 295 bytes in 427 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:55:34.144443 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a9bc3b-dc9b-4b66-b6f1-1fc1101577d3 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:34.182355 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a9bc3b-dc9b-4b66-b6f1-1fc1101577d3 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:34.211924 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c70331f-1a75-40d1-bdc0-51cecf62dd76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:34.212618 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c70331f-1a75-40d1-bdc0-51cecf62dd76 admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:34.212901 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c70331f-1a75-40d1-bdc0-51cecf62dd76 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:34.213171 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c70331f-1a75-40d1-bdc0-51cecf62dd76 admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:34.213946 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c70331f-1a75-40d1-bdc0-51cecf62dd76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:34.215653 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c70331f-1a75-40d1-bdc0-51cecf62dd76 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:34.236069 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ef135f-50b2-4f00-b48d-5660f8a61090 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:34.245918 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 828/1664] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:55:34 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:55:34.248559 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ef135f-50b2-4f00-b48d-5660f8a61090 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:34.255949 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fb07dc-f386-41ac-98ad-ea32cf94836e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:34.256666 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fb07dc-f386-41ac-98ad-ea32cf94836e admin admin] REQUEST_METHOD: `PUT` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:34.257095 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fb07dc-f386-41ac-98ad-ea32cf94836e admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:34.260770 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fb07dc-f386-41ac-98ad-ea32cf94836e admin admin] PATH_INFO: `/v3/projects/fdc545d87c8344379548da5f65552dad/users/500945cbdd7f4812adf7c5ae48a42caf/roles/b870cd1acaa744e1ae1d58b9ac283821` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:34.292900 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fb07dc-f386-41ac-98ad-ea32cf94836e admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdc545d87c8344379548da5f65552dad, user_id=500945cbdd7f4812adf7c5ae48a42caf, role_id=b870cd1acaa744e1ae1d58b9ac283821)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:34.297154 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fb07dc-f386-41ac-98ad-ea32cf94836e admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:34.400424 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 837/1665] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 08:55:34 2022] PUT /identity/v3/projects/fdc545d87c8344379548da5f65552dad/users/500945cbdd7f4812adf7c5ae48a42caf/roles/b870cd1acaa744e1ae1d58b9ac283821 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:55:34.404776 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c70331f-1a75-40d1-bdc0-51cecf62dd76 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:34.417185 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c70331f-1a75-40d1-bdc0-51cecf62dd76 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:34.508576 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20deb580-1082-45a0-bde8-95ac78521ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:34.509434 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20deb580-1082-45a0-bde8-95ac78521ff7 admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:34.520571 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:34.520571 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20deb580-1082-45a0-bde8-95ac78521ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:34.520571 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20deb580-1082-45a0-bde8-95ac78521ff7 admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:34.520571 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20deb580-1082-45a0-bde8-95ac78521ff7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:34.520571 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20deb580-1082-45a0-bde8-95ac78521ff7 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:34.539331 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 829/1666] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:55:34 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:55:34.543036 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fb07dc-f386-41ac-98ad-ea32cf94836e admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:34.588362 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fb07dc-f386-41ac-98ad-ea32cf94836e admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:34.597271 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd1f8a6-7f6d-4ed9-9cfe-274f74acd502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:34.598567 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd1f8a6-7f6d-4ed9-9cfe-274f74acd502 admin admin] REQUEST_METHOD: `PUT` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:34.599025 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd1f8a6-7f6d-4ed9-9cfe-274f74acd502 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:34.599528 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd1f8a6-7f6d-4ed9-9cfe-274f74acd502 admin admin] PATH_INFO: `/v3/projects/fdc545d87c8344379548da5f65552dad/users/500945cbdd7f4812adf7c5ae48a42caf/roles/b870cd1acaa744e1ae1d58b9ac283821` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:34.612090 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd1f8a6-7f6d-4ed9-9cfe-274f74acd502 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdc545d87c8344379548da5f65552dad, user_id=500945cbdd7f4812adf7c5ae48a42caf, role_id=b870cd1acaa744e1ae1d58b9ac283821)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:34.615993 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd1f8a6-7f6d-4ed9-9cfe-274f74acd502 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:34.650962 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 838/1667] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 08:55:34 2022] PUT /identity/v3/projects/fdc545d87c8344379548da5f65552dad/users/500945cbdd7f4812adf7c5ae48a42caf/roles/b870cd1acaa744e1ae1d58b9ac283821 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:55:34.663319 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da113e86-3b13-4ced-881c-8771118bfe27 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:34.663855 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da113e86-3b13-4ced-881c-8771118bfe27 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:34.664289 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da113e86-3b13-4ced-881c-8771118bfe27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:34.732281 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-da113e86-3b13-4ced-881c-8771118bfe27 None None] MFA Rules not processed for user `500945cbdd7f4812adf7c5ae48a42caf`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:34.733595 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-da113e86-3b13-4ced-881c-8771118bfe27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:34.912024 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 830/1668] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:55:34 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:34.925586 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd1f8a6-7f6d-4ed9-9cfe-274f74acd502 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:34.927864 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-bbd1f8a6-7f6d-4ed9-9cfe-274f74acd502 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:35.061554 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd1f8a6-7f6d-4ed9-9cfe-274f74acd502 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:35.074835 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4f949c-2cd0-4e9f-88cf-6831d208e31f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbb9ac667f04c35a63b2689df18efed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:35.075736 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4f949c-2cd0-4e9f-88cf-6831d208e31f service cinder] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:35.076154 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4f949c-2cd0-4e9f-88cf-6831d208e31f service cinder] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:35.076798 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4f949c-2cd0-4e9f-88cf-6831d208e31f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:35.079249 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-4e4f949c-2cd0-4e9f-88cf-6831d208e31f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:35.091020 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da113e86-3b13-4ced-881c-8771118bfe27 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:35.093340 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-da113e86-3b13-4ced-881c-8771118bfe27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:35.211348 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4f949c-2cd0-4e9f-88cf-6831d208e31f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:35.213144 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4f949c-2cd0-4e9f-88cf-6831d208e31f service cinder] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:35.226280 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 839/1669] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:55:34 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:55:35.249941 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da113e86-3b13-4ced-881c-8771118bfe27 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:35.270667 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d5ceca-bb12-43a8-a7a3-acc5823e4ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:35.275224 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d5ceca-bb12-43a8-a7a3-acc5823e4ece admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:35.275224 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d5ceca-bb12-43a8-a7a3-acc5823e4ece admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:35.275224 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d5ceca-bb12-43a8-a7a3-acc5823e4ece admin admin] PATH_INFO: `/v3/projects/c469c437085945c4a7adb6e1aa148304` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:35.275224 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d5ceca-bb12-43a8-a7a3-acc5823e4ece admin admin] RBAC: Authorizing `identity:delete_project(project_id=c469c437085945c4a7adb6e1aa148304)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:35.278307 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d5ceca-bb12-43a8-a7a3-acc5823e4ece admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:35.496702 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-a4d5ceca-bb12-43a8-a7a3-acc5823e4ece admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c469c437085945c4a7adb6e1aa148304', 'request_id': 'req-a4d5ceca-bb12-43a8-a7a3-acc5823e4ece'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:35.628165 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-a4d5ceca-bb12-43a8-a7a3-acc5823e4ece admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c469c437085945c4a7adb6e1aa148304', 'request_id': 'req-a4d5ceca-bb12-43a8-a7a3-acc5823e4ece'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:35.641156 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-a4d5ceca-bb12-43a8-a7a3-acc5823e4ece admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c469c437085945c4a7adb6e1aa148304', 'request_id': 'req-a4d5ceca-bb12-43a8-a7a3-acc5823e4ece'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:35.693547 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-a4d5ceca-bb12-43a8-a7a3-acc5823e4ece admin admin] The token cache is being invalidate because project c469c437085945c4a7adb6e1aa148304 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:55:35.693881 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-a4d5ceca-bb12-43a8-a7a3-acc5823e4ece admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:35.696067 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-a4d5ceca-bb12-43a8-a7a3-acc5823e4ece admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:35.703043 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 831/1670] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 08:55:35 2022] DELETE /identity/v3/projects/c469c437085945c4a7adb6e1aa148304 => generated 0 bytes in 613 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:55:35.708825 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bda598f-fbf9-41f8-beeb-38e6db8ac309 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:35.709239 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bda598f-fbf9-41f8-beeb-38e6db8ac309 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:35.709588 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bda598f-fbf9-41f8-beeb-38e6db8ac309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:35.732349 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-2bda598f-fbf9-41f8-beeb-38e6db8ac309 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:35.733943 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-2bda598f-fbf9-41f8-beeb-38e6db8ac309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:35.879147 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 840/1671] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:55:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:35.881361 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1656d3c9-c932-4347-8860-4cade84383b3 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:35.881860 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1656d3c9-c932-4347-8860-4cade84383b3 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:35.882288 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1656d3c9-c932-4347-8860-4cade84383b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:35.924509 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=84,keystone:INSERT=2,keystone:DELETE=14 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:35.925602 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-1656d3c9-c932-4347-8860-4cade84383b3 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:35.929200 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-1656d3c9-c932-4347-8860-4cade84383b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:35.952304 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 832/1672] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:55:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:35.956987 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bda598f-fbf9-41f8-beeb-38e6db8ac309 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:35.971208 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bda598f-fbf9-41f8-beeb-38e6db8ac309 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:35.985379 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5400fe5b-23e5-44f3-bf05-b424cd250774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:35.986356 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5400fe5b-23e5-44f3-bf05-b424cd250774 admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:35.986734 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5400fe5b-23e5-44f3-bf05-b424cd250774 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:35.987108 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5400fe5b-23e5-44f3-bf05-b424cd250774 admin admin] PATH_INFO: `/v3/domains` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:35.987911 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5400fe5b-23e5-44f3-bf05-b424cd250774 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:55:35.988367 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5400fe5b-23e5-44f3-bf05-b424cd250774 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:35.992338 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5400fe5b-23e5-44f3-bf05-b424cd250774 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:36.023224 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 841/1673] 10.222.0.55 () {58 vars in 1153 bytes} [Wed Aug 24 08:55:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:55:36.028017 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1656d3c9-c932-4347-8860-4cade84383b3 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:36.045294 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1656d3c9-c932-4347-8860-4cade84383b3 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:36.051038 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00051e7-2aeb-4224-b49a-f5e7a9ecdbe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:36.052160 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00051e7-2aeb-4224-b49a-f5e7a9ecdbe1 admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:36.052730 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00051e7-2aeb-4224-b49a-f5e7a9ecdbe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:36.053214 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00051e7-2aeb-4224-b49a-f5e7a9ecdbe1 admin admin] PATH_INFO: `/v3/projects` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:36.054221 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00051e7-2aeb-4224-b49a-f5e7a9ecdbe1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:36.056628 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00051e7-2aeb-4224-b49a-f5e7a9ecdbe1 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:36.125140 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:36.223984 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 833/1674] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 08:55:36 2022] POST /identity/v3/projects => generated 379 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:55:36.231205 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5400fe5b-23e5-44f3-bf05-b424cd250774 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:36.245968 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5400fe5b-23e5-44f3-bf05-b424cd250774 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:36.383849 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b56fe1-8b6f-4791-9dc1-4123d6fad943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:36.388048 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b56fe1-8b6f-4791-9dc1-4123d6fad943 admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:36.388048 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b56fe1-8b6f-4791-9dc1-4123d6fad943 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:36.388048 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b56fe1-8b6f-4791-9dc1-4123d6fad943 admin admin] PATH_INFO: `/v3/users` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:36.389284 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b56fe1-8b6f-4791-9dc1-4123d6fad943 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:36.397780 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b56fe1-8b6f-4791-9dc1-4123d6fad943 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:36.526584 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 842/1675] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 08:55:36 2022] POST /identity/v3/users => generated 290 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:55:36.534139 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00051e7-2aeb-4224-b49a-f5e7a9ecdbe1 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:36.563778 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00051e7-2aeb-4224-b49a-f5e7a9ecdbe1 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:36.570614 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61433e3d-7645-403f-8fe2-4b5331762a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:36.571745 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61433e3d-7645-403f-8fe2-4b5331762a54 admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:36.573230 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61433e3d-7645-403f-8fe2-4b5331762a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:36.573892 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61433e3d-7645-403f-8fe2-4b5331762a54 admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:36.575222 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61433e3d-7645-403f-8fe2-4b5331762a54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:36.577424 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61433e3d-7645-403f-8fe2-4b5331762a54 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:36.598629 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 834/1676] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:55:36 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:55:36.602695 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b56fe1-8b6f-4791-9dc1-4123d6fad943 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:36.616732 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b56fe1-8b6f-4791-9dc1-4123d6fad943 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:36.623493 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982c2669-2922-4a16-80d1-1b0fecfaa157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:36.630491 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982c2669-2922-4a16-80d1-1b0fecfaa157 admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:36.630491 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982c2669-2922-4a16-80d1-1b0fecfaa157 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:36.630491 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982c2669-2922-4a16-80d1-1b0fecfaa157 admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:36.630491 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982c2669-2922-4a16-80d1-1b0fecfaa157 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:36.630491 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982c2669-2922-4a16-80d1-1b0fecfaa157 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:36.649324 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 843/1677] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:55:36 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:55:36.653923 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61433e3d-7645-403f-8fe2-4b5331762a54 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:36.669278 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61433e3d-7645-403f-8fe2-4b5331762a54 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:36.675554 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ee076e-962a-4896-bd30-5e12b2a95797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:36.676476 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ee076e-962a-4896-bd30-5e12b2a95797 admin admin] REQUEST_METHOD: `PUT` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:36.676992 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ee076e-962a-4896-bd30-5e12b2a95797 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:36.677473 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ee076e-962a-4896-bd30-5e12b2a95797 admin admin] PATH_INFO: `/v3/projects/52c5129532834329a2f6356889fc3aa6/users/4cee68c808ff4d798c8ce42941b2314e/roles/b674fd0a592e4e0cada811bba5ceea4d` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:36.718484 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ee076e-962a-4896-bd30-5e12b2a95797 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52c5129532834329a2f6356889fc3aa6, user_id=4cee68c808ff4d798c8ce42941b2314e, role_id=b674fd0a592e4e0cada811bba5ceea4d)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:36.734038 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ee076e-962a-4896-bd30-5e12b2a95797 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:36.895711 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 835/1678] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 08:55:36 2022] PUT /identity/v3/projects/52c5129532834329a2f6356889fc3aa6/users/4cee68c808ff4d798c8ce42941b2314e/roles/b674fd0a592e4e0cada811bba5ceea4d => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:55:36.909201 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce3c19c-94ed-4fd2-9bde-3e4ef71da333 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:36.909645 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce3c19c-94ed-4fd2-9bde-3e4ef71da333 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:36.910022 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce3c19c-94ed-4fd2-9bde-3e4ef71da333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:36.997049 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-0ce3c19c-94ed-4fd2-9bde-3e4ef71da333 None None] MFA Rules not processed for user `4cee68c808ff4d798c8ce42941b2314e`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:36.998641 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-0ce3c19c-94ed-4fd2-9bde-3e4ef71da333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:37.176229 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 844/1679] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:55:36 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:37.177224 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ee076e-962a-4896-bd30-5e12b2a95797 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:37.269874 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805369fc-2e0b-446c-9156-0426f2b53884 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:37.270426 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805369fc-2e0b-446c-9156-0426f2b53884 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:37.270686 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805369fc-2e0b-446c-9156-0426f2b53884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:37.348720 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-805369fc-2e0b-446c-9156-0426f2b53884 None None] MFA Rules not processed for user `f2b2a9bcb9884fadaf78086b0cc4ea6d`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:37.357663 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-805369fc-2e0b-446c-9156-0426f2b53884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:37.417143 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ee076e-962a-4896-bd30-5e12b2a95797 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:37.423986 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23b3d75-d80e-4656-ad9e-ba9327d5ddf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:37.424644 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b3d75-d80e-4656-ad9e-ba9327d5ddf5 admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:37.424926 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b3d75-d80e-4656-ad9e-ba9327d5ddf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:37.425192 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b3d75-d80e-4656-ad9e-ba9327d5ddf5 admin admin] PATH_INFO: `/v3/projects` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:37.425831 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23b3d75-d80e-4656-ad9e-ba9327d5ddf5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:37.445404 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23b3d75-d80e-4656-ad9e-ba9327d5ddf5 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:37.595218 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 845/1680] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:55:37 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:37.632006 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805369fc-2e0b-446c-9156-0426f2b53884 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:37.639032 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 836/1681] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 08:55:37 2022] POST /identity/v3/projects => generated 381 bytes in 462 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:55:37.647144 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:37.649610 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805369fc-2e0b-446c-9156-0426f2b53884 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:37.757424 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=91,keystone:DELETE=3,keystone:INSERT=6 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:37.811013 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb5a603-511f-4785-9911-376e9bcac157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:37.820394 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb5a603-511f-4785-9911-376e9bcac157 admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:37.820394 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb5a603-511f-4785-9911-376e9bcac157 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:37.820394 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb5a603-511f-4785-9911-376e9bcac157 admin admin] PATH_INFO: `/v3/users` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:37.820394 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb5a603-511f-4785-9911-376e9bcac157 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:37.823998 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb5a603-511f-4785-9911-376e9bcac157 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:37.972908 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 846/1682] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 08:55:37 2022] POST /identity/v3/users => generated 297 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:55:37.978162 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23b3d75-d80e-4656-ad9e-ba9327d5ddf5 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:38.007372 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23b3d75-d80e-4656-ad9e-ba9327d5ddf5 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:38.047232 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42829a51-c1c0-4f17-be05-5a31c480286d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:38.047232 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42829a51-c1c0-4f17-be05-5a31c480286d admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:38.047232 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42829a51-c1c0-4f17-be05-5a31c480286d admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:38.047232 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42829a51-c1c0-4f17-be05-5a31c480286d admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:38.047918 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42829a51-c1c0-4f17-be05-5a31c480286d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:38.047918 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42829a51-c1c0-4f17-be05-5a31c480286d admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:38.078394 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 837/1683] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:55:37 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:55:38.081570 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb5a603-511f-4785-9911-376e9bcac157 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:38.096790 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb5a603-511f-4785-9911-376e9bcac157 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:38.125386 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699b388b-fe4c-49b2-9dd3-f70944f9161f None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:38.125725 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699b388b-fe4c-49b2-9dd3-f70944f9161f None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:38.125993 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699b388b-fe4c-49b2-9dd3-f70944f9161f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:38.133764 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4beb92-4944-4406-8b9b-fe78a19224d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:38.137227 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4beb92-4944-4406-8b9b-fe78a19224d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:38.137227 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4beb92-4944-4406-8b9b-fe78a19224d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:38.137227 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4beb92-4944-4406-8b9b-fe78a19224d8 admin admin] PATH_INFO: `/v3/projects/c4169dc29f4f4c1fbbb88cdd84bcd649/users/b734b53c08944802929c767dd93e39cc/roles/b870cd1acaa744e1ae1d58b9ac283821` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:38.201878 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4beb92-4944-4406-8b9b-fe78a19224d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4169dc29f4f4c1fbbb88cdd84bcd649, user_id=b734b53c08944802929c767dd93e39cc, role_id=b870cd1acaa744e1ae1d58b9ac283821)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:38.207638 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4beb92-4944-4406-8b9b-fe78a19224d8 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:38.219231 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-699b388b-fe4c-49b2-9dd3-f70944f9161f None None] MFA Rules not processed for user `718a0dfdef714e358a74ad8e424b1b8a`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:38.221422 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-699b388b-fe4c-49b2-9dd3-f70944f9161f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:38.303121 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 847/1684] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 08:55:38 2022] PUT /identity/v3/projects/c4169dc29f4f4c1fbbb88cdd84bcd649/users/b734b53c08944802929c767dd93e39cc/roles/b870cd1acaa744e1ae1d58b9ac283821 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:55:38.307254 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4beb92-4944-4406-8b9b-fe78a19224d8 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:38.437121 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 838/1685] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:55:38 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:38.448090 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb42500-38dc-490c-9b70-724667e3b888 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:38.448500 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb42500-38dc-490c-9b70-724667e3b888 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:38.448926 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb42500-38dc-490c-9b70-724667e3b888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:38.480932 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-acb42500-38dc-490c-9b70-724667e3b888 None None] MFA Rules not processed for user `718a0dfdef714e358a74ad8e424b1b8a`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:38.486012 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-acb42500-38dc-490c-9b70-724667e3b888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:38.498723 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4beb92-4944-4406-8b9b-fe78a19224d8 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:38.511487 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9ee194-6c05-42ec-8fae-3d6ff9350cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:38.513140 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9ee194-6c05-42ec-8fae-3d6ff9350cc9 admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:38.514061 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9ee194-6c05-42ec-8fae-3d6ff9350cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:38.516687 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9ee194-6c05-42ec-8fae-3d6ff9350cc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:38.518376 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9ee194-6c05-42ec-8fae-3d6ff9350cc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:38.524436 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9ee194-6c05-42ec-8fae-3d6ff9350cc9 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:38.553196 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 848/1686] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:55:38 2022] GET /identity/v3/roles => generated 1420 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:55:38.557477 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9ee194-6c05-42ec-8fae-3d6ff9350cc9 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:38.581852 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9ee194-6c05-42ec-8fae-3d6ff9350cc9 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:38.588657 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 839/1687] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:55:38 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:38.595601 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb42500-38dc-490c-9b70-724667e3b888 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:38.597722 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-acb42500-38dc-490c-9b70-724667e3b888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:38.598579 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7d0b96-57bd-4d88-8167-0736d68c789e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:38.599272 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7d0b96-57bd-4d88-8167-0736d68c789e admin admin] REQUEST_METHOD: `PUT` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:38.600235 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7d0b96-57bd-4d88-8167-0736d68c789e admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:38.600592 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7d0b96-57bd-4d88-8167-0736d68c789e admin admin] PATH_INFO: `/v3/projects/c4169dc29f4f4c1fbbb88cdd84bcd649/users/b734b53c08944802929c767dd93e39cc/roles/b870cd1acaa744e1ae1d58b9ac283821` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:38.602592 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7d0b96-57bd-4d88-8167-0736d68c789e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4169dc29f4f4c1fbbb88cdd84bcd649, user_id=b734b53c08944802929c767dd93e39cc, role_id=b870cd1acaa744e1ae1d58b9ac283821)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:38.605499 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7d0b96-57bd-4d88-8167-0736d68c789e admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:38.620059 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 849/1688] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 08:55:38 2022] PUT /identity/v3/projects/c4169dc29f4f4c1fbbb88cdd84bcd649/users/b734b53c08944802929c767dd93e39cc/roles/b870cd1acaa744e1ae1d58b9ac283821 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:55:38.635108 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cf0655-7dce-4bec-a1cd-91a09a890392 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:38.635396 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cf0655-7dce-4bec-a1cd-91a09a890392 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:38.635672 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cf0655-7dce-4bec-a1cd-91a09a890392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:38.714184 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-66cf0655-7dce-4bec-a1cd-91a09a890392 None None] MFA Rules not processed for user `b734b53c08944802929c767dd93e39cc`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:38.715883 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-66cf0655-7dce-4bec-a1cd-91a09a890392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:38.798046 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb42500-38dc-490c-9b70-724667e3b888 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:38.835191 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 850/1689] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:55:38 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:55:38.856133 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cf0655-7dce-4bec-a1cd-91a09a890392 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:38.858388 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-66cf0655-7dce-4bec-a1cd-91a09a890392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:38.942129 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11230ba-c470-4c66-a96c-4283689f862b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17da357e555420799ee681bffbcc268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:38.942765 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11230ba-c470-4c66-a96c-4283689f862b service nova] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:38.943790 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11230ba-c470-4c66-a96c-4283689f862b service nova] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:38.948210 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11230ba-c470-4c66-a96c-4283689f862b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:38.995112 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cf0655-7dce-4bec-a1cd-91a09a890392 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:39.001275 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f06a867-e10b-4c04-a20f-9e0c1648e620 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbb9ac667f04c35a63b2689df18efed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:39.002033 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f06a867-e10b-4c04-a20f-9e0c1648e620 service cinder] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:39.002443 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f06a867-e10b-4c04-a20f-9e0c1648e620 service cinder] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:39.003008 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f06a867-e10b-4c04-a20f-9e0c1648e620 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:39.005352 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:39.016767 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f06a867-e10b-4c04-a20f-9e0c1648e620 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:39.018792 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f06a867-e10b-4c04-a20f-9e0c1648e620 service cinder] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:39.034718 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 851/1690] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:55:38 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:55:39.097455 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11230ba-c470-4c66-a96c-4283689f862b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:39.099542 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11230ba-c470-4c66-a96c-4283689f862b service nova] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:39.143011 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 840/1691] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:55:38 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 548 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:55:39.257535 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:40.106087 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:40.276632 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:40.309688 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b4ce48-f155-4542-87ba-764400766b13 None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:40.316636 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b4ce48-f155-4542-87ba-764400766b13 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:40.316636 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b4ce48-f155-4542-87ba-764400766b13 None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:40.316636 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 852/1692] 10.222.0.55 () {54 vars in 912 bytes} [Wed Aug 24 08:55:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:55:40.327164 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d16d9cf-6ba5-4549-a6d1-899a82504ccc None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:40.327815 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d16d9cf-6ba5-4549-a6d1-899a82504ccc None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:40.328273 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d16d9cf-6ba5-4549-a6d1-899a82504ccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:40.450672 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-8d16d9cf-6ba5-4549-a6d1-899a82504ccc None None] MFA Rules not processed for user `28b58388d1664510b0b507323bca5ae8`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:40.454431 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-8d16d9cf-6ba5-4549-a6d1-899a82504ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:40.615512 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 841/1693] 10.222.0.55 () {58 vars in 1009 bytes} [Wed Aug 24 08:55:40 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:40.669018 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b4ce48-f155-4542-87ba-764400766b13 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:40.705518 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b4ce48-f155-4542-87ba-764400766b13 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:40.738039 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0951246a-8710-43c8-ac21-8d53777d81f8 tempest-VolumeQuotaClassesTest-1677038873 tempest-VolumeQuotaClassesTest-1677038873-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b734b53c08944802929c767dd93e39cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4169dc29f4f4c1fbbb88cdd84bcd649', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:40.740518 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0951246a-8710-43c8-ac21-8d53777d81f8 tempest-VolumeQuotaClassesTest-1677038873 tempest-VolumeQuotaClassesTest-1677038873-project-admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:40.743031 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0951246a-8710-43c8-ac21-8d53777d81f8 tempest-VolumeQuotaClassesTest-1677038873 tempest-VolumeQuotaClassesTest-1677038873-project-admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:40.743341 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0951246a-8710-43c8-ac21-8d53777d81f8 tempest-VolumeQuotaClassesTest-1677038873 tempest-VolumeQuotaClassesTest-1677038873-project-admin] PATH_INFO: `/v3/domains` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:40.748871 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0951246a-8710-43c8-ac21-8d53777d81f8 tempest-VolumeQuotaClassesTest-1677038873 tempest-VolumeQuotaClassesTest-1677038873-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:55:40.749139 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0951246a-8710-43c8-ac21-8d53777d81f8 tempest-VolumeQuotaClassesTest-1677038873 tempest-VolumeQuotaClassesTest-1677038873-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:40.756079 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0951246a-8710-43c8-ac21-8d53777d81f8 tempest-VolumeQuotaClassesTest-1677038873 tempest-VolumeQuotaClassesTest-1677038873-project-admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:40.812975 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 853/1694] 10.222.0.55 () {58 vars in 1153 bytes} [Wed Aug 24 08:55:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:55:40.819136 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d16d9cf-6ba5-4549-a6d1-899a82504ccc None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:40.844730 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d16d9cf-6ba5-4549-a6d1-899a82504ccc None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:40.865993 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974cbd21-457b-4f6e-8d15-7a2f8bc25653 tempest-VolumeQuotaClassesTest-1677038873 tempest-VolumeQuotaClassesTest-1677038873-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b734b53c08944802929c767dd93e39cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4169dc29f4f4c1fbbb88cdd84bcd649', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:40.866629 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974cbd21-457b-4f6e-8d15-7a2f8bc25653 tempest-VolumeQuotaClassesTest-1677038873 tempest-VolumeQuotaClassesTest-1677038873-project-admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:40.866924 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974cbd21-457b-4f6e-8d15-7a2f8bc25653 tempest-VolumeQuotaClassesTest-1677038873 tempest-VolumeQuotaClassesTest-1677038873-project-admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:40.867204 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974cbd21-457b-4f6e-8d15-7a2f8bc25653 tempest-VolumeQuotaClassesTest-1677038873 tempest-VolumeQuotaClassesTest-1677038873-project-admin] PATH_INFO: `/v3/projects` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:40.867970 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974cbd21-457b-4f6e-8d15-7a2f8bc25653 tempest-VolumeQuotaClassesTest-1677038873 tempest-VolumeQuotaClassesTest-1677038873-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:40.869863 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974cbd21-457b-4f6e-8d15-7a2f8bc25653 tempest-VolumeQuotaClassesTest-1677038873 tempest-VolumeQuotaClassesTest-1677038873-project-admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:40.946135 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0951246a-8710-43c8-ac21-8d53777d81f8 tempest-VolumeQuotaClassesTest-1677038873 tempest-VolumeQuotaClassesTest-1677038873-project-admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:40.948057 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-0951246a-8710-43c8-ac21-8d53777d81f8 tempest-VolumeQuotaClassesTest-1677038873 tempest-VolumeQuotaClassesTest-1677038873-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:40.963605 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0951246a-8710-43c8-ac21-8d53777d81f8 tempest-VolumeQuotaClassesTest-1677038873 tempest-VolumeQuotaClassesTest-1677038873-project-admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:40.969633 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3962f3-f5f9-415a-86cf-712d519dbc78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:40.972406 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3962f3-f5f9-415a-86cf-712d519dbc78 service neutron] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:40.972406 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3962f3-f5f9-415a-86cf-712d519dbc78 service neutron] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:40.972406 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3962f3-f5f9-415a-86cf-712d519dbc78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:40.984096 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3962f3-f5f9-415a-86cf-712d519dbc78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:40.990684 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3962f3-f5f9-415a-86cf-712d519dbc78 service neutron] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:41.017269 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 854/1695] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:55:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:55:41.033023 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 842/1696] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 08:55:40 2022] POST /identity/v3/projects => generated 353 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:55:41.037603 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3962f3-f5f9-415a-86cf-712d519dbc78 service neutron] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:41.055150 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3962f3-f5f9-415a-86cf-712d519dbc78 service neutron] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:41.203777 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5a40b3-bde2-4c4d-8ef7-90bf632242e4 tempest-VolumeQuotaClassesTest-1677038873 tempest-VolumeQuotaClassesTest-1677038873-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b734b53c08944802929c767dd93e39cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4169dc29f4f4c1fbbb88cdd84bcd649', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:41.205359 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5a40b3-bde2-4c4d-8ef7-90bf632242e4 tempest-VolumeQuotaClassesTest-1677038873 tempest-VolumeQuotaClassesTest-1677038873-project-admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:41.205657 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5a40b3-bde2-4c4d-8ef7-90bf632242e4 tempest-VolumeQuotaClassesTest-1677038873 tempest-VolumeQuotaClassesTest-1677038873-project-admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:41.205945 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5a40b3-bde2-4c4d-8ef7-90bf632242e4 tempest-VolumeQuotaClassesTest-1677038873 tempest-VolumeQuotaClassesTest-1677038873-project-admin] PATH_INFO: `/v3/domains` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:41.206766 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5a40b3-bde2-4c4d-8ef7-90bf632242e4 tempest-VolumeQuotaClassesTest-1677038873 tempest-VolumeQuotaClassesTest-1677038873-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:55:41.207090 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5a40b3-bde2-4c4d-8ef7-90bf632242e4 tempest-VolumeQuotaClassesTest-1677038873 tempest-VolumeQuotaClassesTest-1677038873-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:41.209591 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5a40b3-bde2-4c4d-8ef7-90bf632242e4 tempest-VolumeQuotaClassesTest-1677038873 tempest-VolumeQuotaClassesTest-1677038873-project-admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:41.232144 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 855/1697] 10.222.0.55 () {58 vars in 1153 bytes} [Wed Aug 24 08:55:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:55:41.272484 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:41.309342 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974cbd21-457b-4f6e-8d15-7a2f8bc25653 tempest-VolumeQuotaClassesTest-1677038873 tempest-VolumeQuotaClassesTest-1677038873-project-admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:41.323117 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974cbd21-457b-4f6e-8d15-7a2f8bc25653 tempest-VolumeQuotaClassesTest-1677038873 tempest-VolumeQuotaClassesTest-1677038873-project-admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:41.337274 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27cee1b-7b39-4c2a-9d5a-a6841497906f tempest-VolumeQuotaClassesTest-1677038873 tempest-VolumeQuotaClassesTest-1677038873-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b734b53c08944802929c767dd93e39cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4169dc29f4f4c1fbbb88cdd84bcd649', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:41.338131 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27cee1b-7b39-4c2a-9d5a-a6841497906f tempest-VolumeQuotaClassesTest-1677038873 tempest-VolumeQuotaClassesTest-1677038873-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:41.338574 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27cee1b-7b39-4c2a-9d5a-a6841497906f tempest-VolumeQuotaClassesTest-1677038873 tempest-VolumeQuotaClassesTest-1677038873-project-admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:41.338997 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27cee1b-7b39-4c2a-9d5a-a6841497906f tempest-VolumeQuotaClassesTest-1677038873 tempest-VolumeQuotaClassesTest-1677038873-project-admin] PATH_INFO: `/v3/projects/e27da387afc64e2b9fcf3f538d7382e0` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:41.340040 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:41.341369 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27cee1b-7b39-4c2a-9d5a-a6841497906f tempest-VolumeQuotaClassesTest-1677038873 tempest-VolumeQuotaClassesTest-1677038873-project-admin] RBAC: Authorizing `identity:delete_project(project_id=e27da387afc64e2b9fcf3f538d7382e0)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:41.347007 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27cee1b-7b39-4c2a-9d5a-a6841497906f tempest-VolumeQuotaClassesTest-1677038873 tempest-VolumeQuotaClassesTest-1677038873-project-admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:41.530722 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-e27cee1b-7b39-4c2a-9d5a-a6841497906f tempest-VolumeQuotaClassesTest-1677038873 tempest-VolumeQuotaClassesTest-1677038873-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e27da387afc64e2b9fcf3f538d7382e0', 'request_id': 'req-e27cee1b-7b39-4c2a-9d5a-a6841497906f'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:41.645253 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-e27cee1b-7b39-4c2a-9d5a-a6841497906f tempest-VolumeQuotaClassesTest-1677038873 tempest-VolumeQuotaClassesTest-1677038873-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e27da387afc64e2b9fcf3f538d7382e0', 'request_id': 'req-e27cee1b-7b39-4c2a-9d5a-a6841497906f'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:41.653283 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-e27cee1b-7b39-4c2a-9d5a-a6841497906f tempest-VolumeQuotaClassesTest-1677038873 tempest-VolumeQuotaClassesTest-1677038873-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e27da387afc64e2b9fcf3f538d7382e0', 'request_id': 'req-e27cee1b-7b39-4c2a-9d5a-a6841497906f'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:41.680316 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-e27cee1b-7b39-4c2a-9d5a-a6841497906f tempest-VolumeQuotaClassesTest-1677038873 tempest-VolumeQuotaClassesTest-1677038873-project-admin] The token cache is being invalidate because project e27da387afc64e2b9fcf3f538d7382e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:55:41.680728 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-e27cee1b-7b39-4c2a-9d5a-a6841497906f tempest-VolumeQuotaClassesTest-1677038873 tempest-VolumeQuotaClassesTest-1677038873-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:41.682777 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-e27cee1b-7b39-4c2a-9d5a-a6841497906f tempest-VolumeQuotaClassesTest-1677038873 tempest-VolumeQuotaClassesTest-1677038873-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:41.686040 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 843/1698] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 08:55:41 2022] DELETE /identity/v3/projects/e27da387afc64e2b9fcf3f538d7382e0 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:55:42.176268 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b19e84-a596-4cc3-8d3c-dd7bfc33c2c7 None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:42.176827 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b19e84-a596-4cc3-8d3c-dd7bfc33c2c7 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:42.177866 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b19e84-a596-4cc3-8d3c-dd7bfc33c2c7 None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:42.179868 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 856/1699] 10.222.0.55 () {54 vars in 921 bytes} [Wed Aug 24 08:55:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:55:42.201075 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3cf27e-9db1-4e01-a221-494cc02e017c None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:42.201564 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3cf27e-9db1-4e01-a221-494cc02e017c None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:42.202011 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3cf27e-9db1-4e01-a221-494cc02e017c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:42.242698 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-ab3cf27e-9db1-4e01-a221-494cc02e017c None None] MFA Rules not processed for user `cf0c99771661413aaa0f237af6b394ab`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:42.244284 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-ab3cf27e-9db1-4e01-a221-494cc02e017c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:42.387759 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 844/1700] 10.222.0.55 () {58 vars in 1018 bytes} [Wed Aug 24 08:55:42 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:42.458073 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc686b3-0936-4d78-935a-c40235cf3194 None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:42.458633 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc686b3-0936-4d78-935a-c40235cf3194 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:42.459388 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc686b3-0936-4d78-935a-c40235cf3194 None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:42.461188 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 857/1701] 10.222.0.55 () {54 vars in 961 bytes} [Wed Aug 24 08:55:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:55:42.470505 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fb7831-49c5-4cb6-8f3b-f93cb130c5aa None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:42.470931 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fb7831-49c5-4cb6-8f3b-f93cb130c5aa None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:42.471793 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fb7831-49c5-4cb6-8f3b-f93cb130c5aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:42.514336 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-f1fb7831-49c5-4cb6-8f3b-f93cb130c5aa None None] MFA Rules not processed for user `cf0c99771661413aaa0f237af6b394ab`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:42.517505 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-f1fb7831-49c5-4cb6-8f3b-f93cb130c5aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:42.545606 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 845/1702] 10.222.0.55 () {58 vars in 1058 bytes} [Wed Aug 24 08:55:42 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:42.555285 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc686b3-0936-4d78-935a-c40235cf3194 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:42.596754 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc686b3-0936-4d78-935a-c40235cf3194 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:42.612434 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbc5439-f28c-4d90-bcdc-1980fc3e9efb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf0c99771661413aaa0f237af6b394ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:42.613106 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbc5439-f28c-4d90-bcdc-1980fc3e9efb service placement] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:42.613391 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbc5439-f28c-4d90-bcdc-1980fc3e9efb service placement] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:42.613663 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbc5439-f28c-4d90-bcdc-1980fc3e9efb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:42.650607 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbc5439-f28c-4d90-bcdc-1980fc3e9efb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:42.657252 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbc5439-f28c-4d90-bcdc-1980fc3e9efb service placement] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:42.734558 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 858/1703] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:55:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:55:42.991010 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fb7831-49c5-4cb6-8f3b-f93cb130c5aa None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:42.999172 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-f1fb7831-49c5-4cb6-8f3b-f93cb130c5aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:43.060775 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbc5439-f28c-4d90-bcdc-1980fc3e9efb service placement] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:43.081905 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbc5439-f28c-4d90-bcdc-1980fc3e9efb service placement] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:43.107865 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da82e60a-475e-4c2d-8084-a671d8dac1c4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf0c99771661413aaa0f237af6b394ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:43.110164 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da82e60a-475e-4c2d-8084-a671d8dac1c4 service placement] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:43.112907 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da82e60a-475e-4c2d-8084-a671d8dac1c4 service placement] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:43.113376 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da82e60a-475e-4c2d-8084-a671d8dac1c4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:43.124113 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-da82e60a-475e-4c2d-8084-a671d8dac1c4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:43.186663 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da82e60a-475e-4c2d-8084-a671d8dac1c4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:43.190657 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da82e60a-475e-4c2d-8084-a671d8dac1c4 service placement] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:43.223763 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 859/1704] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:55:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:55:43.241070 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fb7831-49c5-4cb6-8f3b-f93cb130c5aa None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:43.241070 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1637ee07-e777-4883-8077-5bd6bbbfd928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:43.241070 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1637ee07-e777-4883-8077-5bd6bbbfd928 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:43.241070 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1637ee07-e777-4883-8077-5bd6bbbfd928 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:43.241439 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1637ee07-e777-4883-8077-5bd6bbbfd928 admin admin] PATH_INFO: `/v3/users/4cee68c808ff4d798c8ce42941b2314e` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:43.242324 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1637ee07-e777-4883-8077-5bd6bbbfd928 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4cee68c808ff4d798c8ce42941b2314e)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:43.259393 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1637ee07-e777-4883-8077-5bd6bbbfd928 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:43.287697 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:43.588219 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-1637ee07-e777-4883-8077-5bd6bbbfd928 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4cee68c808ff4d798c8ce42941b2314e'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:43.611562 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-1637ee07-e777-4883-8077-5bd6bbbfd928 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4cee68c808ff4d798c8ce42941b2314e'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:43.614343 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-1637ee07-e777-4883-8077-5bd6bbbfd928 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4cee68c808ff4d798c8ce42941b2314e'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:43.671592 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-1637ee07-e777-4883-8077-5bd6bbbfd928 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4cee68c808ff4d798c8ce42941b2314e'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:43.938154 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-1637ee07-e777-4883-8077-5bd6bbbfd928 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4cee68c808ff4d798c8ce42941b2314e'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:43.963322 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d393857e-a23f-4f6b-8b65-ae39df49f151 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:43.963864 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 846/1705] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 08:55:42 2022] DELETE /identity/v3/users/4cee68c808ff4d798c8ce42941b2314e => generated 0 bytes in 968 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:55:43.964307 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d393857e-a23f-4f6b-8b65-ae39df49f151 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:43.964771 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d393857e-a23f-4f6b-8b65-ae39df49f151 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:44.014430 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-d393857e-a23f-4f6b-8b65-ae39df49f151 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:44.017408 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-d393857e-a23f-4f6b-8b65-ae39df49f151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:44.164297 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 860/1706] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:55:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:44.185505 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1637ee07-e777-4883-8077-5bd6bbbfd928 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:44.188238 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-1637ee07-e777-4883-8077-5bd6bbbfd928 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:44.361402 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1637ee07-e777-4883-8077-5bd6bbbfd928 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:44.371925 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988aef83-48c1-4902-8f9a-4ba93d8d644e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:44.373427 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988aef83-48c1-4902-8f9a-4ba93d8d644e service neutron] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:44.373930 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988aef83-48c1-4902-8f9a-4ba93d8d644e service neutron] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:44.374808 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988aef83-48c1-4902-8f9a-4ba93d8d644e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:44.402319 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988aef83-48c1-4902-8f9a-4ba93d8d644e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:44.409139 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988aef83-48c1-4902-8f9a-4ba93d8d644e service neutron] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:44.438655 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5d7501-0714-4c53-9ffd-d50169ab73b7 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:44.438946 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5d7501-0714-4c53-9ffd-d50169ab73b7 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:44.439224 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5d7501-0714-4c53-9ffd-d50169ab73b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:44.456768 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 847/1707] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:55:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:55:44.506927 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-fc5d7501-0714-4c53-9ffd-d50169ab73b7 None None] MFA Rules not processed for user `f5d7aee76519492e81b0048edebba36f`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:44.511578 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-fc5d7501-0714-4c53-9ffd-d50169ab73b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:44.523882 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:44.665737 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 861/1708] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:55:44 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:55:44.682802 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988aef83-48c1-4902-8f9a-4ba93d8d644e service neutron] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:44.686043 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-988aef83-48c1-4902-8f9a-4ba93d8d644e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:44.851933 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988aef83-48c1-4902-8f9a-4ba93d8d644e service neutron] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:44.861393 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cf2e0d-7feb-41d6-9b5b-f619e73f1a82 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17da357e555420799ee681bffbcc268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:44.864256 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cf2e0d-7feb-41d6-9b5b-f619e73f1a82 service nova] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:44.865689 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cf2e0d-7feb-41d6-9b5b-f619e73f1a82 service nova] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:44.866182 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cf2e0d-7feb-41d6-9b5b-f619e73f1a82 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:44.886097 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cf2e0d-7feb-41d6-9b5b-f619e73f1a82 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:44.899419 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49cbd43-aae7-4734-a0bc-0e7028f29237 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:44.899973 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49cbd43-aae7-4734-a0bc-0e7028f29237 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:44.900397 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49cbd43-aae7-4734-a0bc-0e7028f29237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:44.902266 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cf2e0d-7feb-41d6-9b5b-f619e73f1a82 service nova] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:44.924829 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 848/1709] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:55:44 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:55:44.940376 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cf2e0d-7feb-41d6-9b5b-f619e73f1a82 service nova] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:44.943105 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-b2cf2e0d-7feb-41d6-9b5b-f619e73f1a82 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:44.953260 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-c49cbd43-aae7-4734-a0bc-0e7028f29237 None None] MFA Rules not processed for user `993697d33b7841668b1eabcdcd6b40f9`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:44.954642 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-c49cbd43-aae7-4734-a0bc-0e7028f29237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:44.996651 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cf2e0d-7feb-41d6-9b5b-f619e73f1a82 service nova] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:45.004432 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e590f3b2-3fdd-4a56-baef-c9a485e23c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:45.008028 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e590f3b2-3fdd-4a56-baef-c9a485e23c2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:45.008028 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e590f3b2-3fdd-4a56-baef-c9a485e23c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:45.008028 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e590f3b2-3fdd-4a56-baef-c9a485e23c2a admin admin] PATH_INFO: `/v3/users/d32c786deac446a4bfb7973701ebd40f` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:45.009488 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e590f3b2-3fdd-4a56-baef-c9a485e23c2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d32c786deac446a4bfb7973701ebd40f)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:45.015047 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e590f3b2-3fdd-4a56-baef-c9a485e23c2a admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:45.127145 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:45.153629 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 862/1710] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:55:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:45.169436 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49cbd43-aae7-4734-a0bc-0e7028f29237 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:45.174213 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-c49cbd43-aae7-4734-a0bc-0e7028f29237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:45.250500 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49cbd43-aae7-4734-a0bc-0e7028f29237 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:45.292426 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946302f9-bac8-44ab-ac62-566738712b76 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17da357e555420799ee681bffbcc268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:45.293115 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946302f9-bac8-44ab-ac62-566738712b76 service nova] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:45.293395 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946302f9-bac8-44ab-ac62-566738712b76 service nova] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:45.293664 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946302f9-bac8-44ab-ac62-566738712b76 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:45.298434 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-e590f3b2-3fdd-4a56-baef-c9a485e23c2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd32c786deac446a4bfb7973701ebd40f'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:45.299554 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-e590f3b2-3fdd-4a56-baef-c9a485e23c2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd32c786deac446a4bfb7973701ebd40f'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:45.300724 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-e590f3b2-3fdd-4a56-baef-c9a485e23c2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd32c786deac446a4bfb7973701ebd40f'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:45.325631 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946302f9-bac8-44ab-ac62-566738712b76 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:45.327028 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946302f9-bac8-44ab-ac62-566738712b76 service nova] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:45.348419 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-e590f3b2-3fdd-4a56-baef-c9a485e23c2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd32c786deac446a4bfb7973701ebd40f'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:45.363521 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 863/1711] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:55:45 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:55:45.474293 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-e590f3b2-3fdd-4a56-baef-c9a485e23c2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd32c786deac446a4bfb7973701ebd40f'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:45.501262 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 849/1712] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 08:55:44 2022] DELETE /identity/v3/users/d32c786deac446a4bfb7973701ebd40f => generated 0 bytes in 561 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:55:46.179054 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fb9b36-4f9c-4d0b-a029-6ffe963d436c None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:46.180017 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fb9b36-4f9c-4d0b-a029-6ffe963d436c None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:46.180498 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fb9b36-4f9c-4d0b-a029-6ffe963d436c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:46.265605 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-17fb9b36-4f9c-4d0b-a029-6ffe963d436c None None] MFA Rules not processed for user `1105970982014891902ebbb9904a57a5`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:46.267558 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-17fb9b36-4f9c-4d0b-a029-6ffe963d436c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:46.329676 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e590f3b2-3fdd-4a56-baef-c9a485e23c2a admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:46.333346 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-e590f3b2-3fdd-4a56-baef-c9a485e23c2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:46.430899 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 864/1713] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:55:46 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:46.448166 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fb9b36-4f9c-4d0b-a029-6ffe963d436c None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:46.450507 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-17fb9b36-4f9c-4d0b-a029-6ffe963d436c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:46.473048 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e590f3b2-3fdd-4a56-baef-c9a485e23c2a admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:46.474993 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:46.479334 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecd19d3-6166-40dd-b1a1-fcee5504aac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:46.480277 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecd19d3-6166-40dd-b1a1-fcee5504aac7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:46.481215 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecd19d3-6166-40dd-b1a1-fcee5504aac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:46.481658 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecd19d3-6166-40dd-b1a1-fcee5504aac7 admin admin] PATH_INFO: `/v3/projects/52c5129532834329a2f6356889fc3aa6` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:46.483355 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecd19d3-6166-40dd-b1a1-fcee5504aac7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=52c5129532834329a2f6356889fc3aa6)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:46.486178 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecd19d3-6166-40dd-b1a1-fcee5504aac7 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:46.602250 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fb9b36-4f9c-4d0b-a029-6ffe963d436c None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:46.627618 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe27d688-4140-40d2-8924-d26b0ea49ad9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17da357e555420799ee681bffbcc268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:46.629741 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe27d688-4140-40d2-8924-d26b0ea49ad9 service nova] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:46.630449 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe27d688-4140-40d2-8924-d26b0ea49ad9 service nova] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:46.630941 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe27d688-4140-40d2-8924-d26b0ea49ad9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:46.645204 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-8ecd19d3-6166-40dd-b1a1-fcee5504aac7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52c5129532834329a2f6356889fc3aa6', 'request_id': 'req-8ecd19d3-6166-40dd-b1a1-fcee5504aac7'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:46.654324 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe27d688-4140-40d2-8924-d26b0ea49ad9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:46.658665 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe27d688-4140-40d2-8924-d26b0ea49ad9 service nova] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:46.671256 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 865/1714] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:55:46 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:55:46.672491 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe27d688-4140-40d2-8924-d26b0ea49ad9 service nova] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:46.674097 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-fe27d688-4140-40d2-8924-d26b0ea49ad9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:46.705101 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe27d688-4140-40d2-8924-d26b0ea49ad9 service nova] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:46.756676 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-8ecd19d3-6166-40dd-b1a1-fcee5504aac7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52c5129532834329a2f6356889fc3aa6', 'request_id': 'req-8ecd19d3-6166-40dd-b1a1-fcee5504aac7'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:46.765985 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-8ecd19d3-6166-40dd-b1a1-fcee5504aac7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52c5129532834329a2f6356889fc3aa6', 'request_id': 'req-8ecd19d3-6166-40dd-b1a1-fcee5504aac7'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:46.792975 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-8ecd19d3-6166-40dd-b1a1-fcee5504aac7 admin admin] The token cache is being invalidate because project 52c5129532834329a2f6356889fc3aa6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:55:46.793344 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-8ecd19d3-6166-40dd-b1a1-fcee5504aac7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:46.805213 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-8ecd19d3-6166-40dd-b1a1-fcee5504aac7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:46.808994 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 850/1715] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 08:55:46 2022] DELETE /identity/v3/projects/52c5129532834329a2f6356889fc3aa6 => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:55:46.813291 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecd19d3-6166-40dd-b1a1-fcee5504aac7 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:46.815611 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-8ecd19d3-6166-40dd-b1a1-fcee5504aac7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:46.903976 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3af8c0-3c36-4a10-a771-3e8986fde102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:46.905102 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3af8c0-3c36-4a10-a771-3e8986fde102 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:46.905392 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3af8c0-3c36-4a10-a771-3e8986fde102 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:46.905709 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3af8c0-3c36-4a10-a771-3e8986fde102 admin admin] PATH_INFO: `/v3/projects/2d8fe1b63acc4365ab4048772c6fccce` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:46.909725 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3af8c0-3c36-4a10-a771-3e8986fde102 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d8fe1b63acc4365ab4048772c6fccce)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:46.914994 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3af8c0-3c36-4a10-a771-3e8986fde102 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:47.027894 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecd19d3-6166-40dd-b1a1-fcee5504aac7 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:47.045401 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8753d2-17cd-4b4f-91a5-603fcde2e01f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:47.046422 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8753d2-17cd-4b4f-91a5-603fcde2e01f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:47.046876 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8753d2-17cd-4b4f-91a5-603fcde2e01f admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:47.047373 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8753d2-17cd-4b4f-91a5-603fcde2e01f admin admin] PATH_INFO: `/v3/users/b734b53c08944802929c767dd93e39cc` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:47.050240 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8753d2-17cd-4b4f-91a5-603fcde2e01f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b734b53c08944802929c767dd93e39cc)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:47.053263 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8753d2-17cd-4b4f-91a5-603fcde2e01f admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:47.104405 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-bc3af8c0-3c36-4a10-a771-3e8986fde102 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d8fe1b63acc4365ab4048772c6fccce', 'request_id': 'req-bc3af8c0-3c36-4a10-a771-3e8986fde102'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:47.135143 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-bc3af8c0-3c36-4a10-a771-3e8986fde102 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d8fe1b63acc4365ab4048772c6fccce', 'request_id': 'req-bc3af8c0-3c36-4a10-a771-3e8986fde102'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:47.291803 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-bc3af8c0-3c36-4a10-a771-3e8986fde102 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d8fe1b63acc4365ab4048772c6fccce', 'request_id': 'req-bc3af8c0-3c36-4a10-a771-3e8986fde102'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:47.314871 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-bc3af8c0-3c36-4a10-a771-3e8986fde102 admin admin] The token cache is being invalidate because project 2d8fe1b63acc4365ab4048772c6fccce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:55:47.316291 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-bc3af8c0-3c36-4a10-a771-3e8986fde102 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:47.317341 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-bc3af8c0-3c36-4a10-a771-3e8986fde102 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:47.323061 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 866/1716] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 08:55:46 2022] DELETE /identity/v3/projects/2d8fe1b63acc4365ab4048772c6fccce => generated 0 bytes in 651 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:55:47.327254 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3af8c0-3c36-4a10-a771-3e8986fde102 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:47.330999 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-bc3af8c0-3c36-4a10-a771-3e8986fde102 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:47.456055 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-9b8753d2-17cd-4b4f-91a5-603fcde2e01f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b734b53c08944802929c767dd93e39cc'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:47.462735 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-9b8753d2-17cd-4b4f-91a5-603fcde2e01f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b734b53c08944802929c767dd93e39cc'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:47.464356 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-9b8753d2-17cd-4b4f-91a5-603fcde2e01f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b734b53c08944802929c767dd93e39cc'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:47.479122 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3af8c0-3c36-4a10-a771-3e8986fde102 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:47.495906 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-9b8753d2-17cd-4b4f-91a5-603fcde2e01f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b734b53c08944802929c767dd93e39cc'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:47.497531 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0c2f99-5fae-4271-9db4-9937e4cdb635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:47.498212 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0c2f99-5fae-4271-9db4-9937e4cdb635 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:47.498495 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0c2f99-5fae-4271-9db4-9937e4cdb635 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:47.498772 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0c2f99-5fae-4271-9db4-9937e4cdb635 admin admin] PATH_INFO: `/v3/users/f2b2a9bcb9884fadaf78086b0cc4ea6d` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:47.501940 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0c2f99-5fae-4271-9db4-9937e4cdb635 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2b2a9bcb9884fadaf78086b0cc4ea6d)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:47.507342 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0c2f99-5fae-4271-9db4-9937e4cdb635 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:47.616998 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-9b8753d2-17cd-4b4f-91a5-603fcde2e01f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b734b53c08944802929c767dd93e39cc'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:47.652671 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:47.658210 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 851/1717] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 08:55:46 2022] DELETE /identity/v3/users/b734b53c08944802929c767dd93e39cc => generated 0 bytes in 840 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:55:47.658210 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8753d2-17cd-4b4f-91a5-603fcde2e01f admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:47.658210 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-9b8753d2-17cd-4b4f-91a5-603fcde2e01f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:47.814465 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-cb0c2f99-5fae-4271-9db4-9937e4cdb635 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2b2a9bcb9884fadaf78086b0cc4ea6d'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:47.815773 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-cb0c2f99-5fae-4271-9db4-9937e4cdb635 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2b2a9bcb9884fadaf78086b0cc4ea6d'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:47.816497 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-cb0c2f99-5fae-4271-9db4-9937e4cdb635 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2b2a9bcb9884fadaf78086b0cc4ea6d'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:47.827193 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8753d2-17cd-4b4f-91a5-603fcde2e01f admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:47.831884 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-cb0c2f99-5fae-4271-9db4-9937e4cdb635 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2b2a9bcb9884fadaf78086b0cc4ea6d'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:47.837471 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e72cf0-7e4e-4351-a393-381f6589f610 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbb9ac667f04c35a63b2689df18efed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:47.841412 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e72cf0-7e4e-4351-a393-381f6589f610 service cinder] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:47.841706 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e72cf0-7e4e-4351-a393-381f6589f610 service cinder] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:47.841993 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e72cf0-7e4e-4351-a393-381f6589f610 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:47.845692 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-47e72cf0-7e4e-4351-a393-381f6589f610 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:47.922915 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-cb0c2f99-5fae-4271-9db4-9937e4cdb635 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2b2a9bcb9884fadaf78086b0cc4ea6d'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:47.952096 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:47.960276 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 867/1718] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 08:55:47 2022] DELETE /identity/v3/users/f2b2a9bcb9884fadaf78086b0cc4ea6d => generated 0 bytes in 630 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:55:48.018657 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e72cf0-7e4e-4351-a393-381f6589f610 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:48.018657 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e72cf0-7e4e-4351-a393-381f6589f610 service cinder] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:48.035632 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 852/1719] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:55:47 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:55:49.139319 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51417abe-51bd-42c3-802b-e620b8e8a106 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:49.140411 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51417abe-51bd-42c3-802b-e620b8e8a106 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:49.140885 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51417abe-51bd-42c3-802b-e620b8e8a106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:49.331723 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-51417abe-51bd-42c3-802b-e620b8e8a106 None None] MFA Rules not processed for user `f652a368b1cc42ae8b6f6c7a6dff6f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:49.333210 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-51417abe-51bd-42c3-802b-e620b8e8a106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:49.374393 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305fb801-91e5-470e-b13e-19298f87b2af None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:49.375632 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305fb801-91e5-470e-b13e-19298f87b2af None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:49.376268 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305fb801-91e5-470e-b13e-19298f87b2af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:49.454732 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-305fb801-91e5-470e-b13e-19298f87b2af None None] MFA Rules not processed for user `f56524e822e641bc9bb11eddfe27f85d`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:49.465458 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-305fb801-91e5-470e-b13e-19298f87b2af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:49.571512 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 868/1720] 10.222.0.55 () {58 vars in 1009 bytes} [Wed Aug 24 08:55:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 433 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:49.643433 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 853/1721] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:55:49 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:49.648339 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51417abe-51bd-42c3-802b-e620b8e8a106 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:49.657271 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-51417abe-51bd-42c3-802b-e620b8e8a106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:49.657427 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf383e8-f74a-4db9-ba57-6614d36a7982 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:49.658821 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf383e8-f74a-4db9-ba57-6614d36a7982 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:49.658821 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf383e8-f74a-4db9-ba57-6614d36a7982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:49.706302 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-aaf383e8-f74a-4db9-ba57-6614d36a7982 None None] MFA Rules not processed for user `f56524e822e641bc9bb11eddfe27f85d`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:49.707489 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-aaf383e8-f74a-4db9-ba57-6614d36a7982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:49.736448 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 854/1722] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:55:49 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:55:49.745749 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:49.755066 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf383e8-f74a-4db9-ba57-6614d36a7982 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:49.756584 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-aaf383e8-f74a-4db9-ba57-6614d36a7982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:49.874179 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51417abe-51bd-42c3-802b-e620b8e8a106 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:49.879924 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05113dd2-b9ab-42da-b395-dfb453a632a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:49.882048 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05113dd2-b9ab-42da-b395-dfb453a632a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:49.882048 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05113dd2-b9ab-42da-b395-dfb453a632a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:49.882048 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05113dd2-b9ab-42da-b395-dfb453a632a5 admin admin] PATH_INFO: `/v3/projects/a353c4943d18429a94aa9d93b67f04db` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:49.882571 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05113dd2-b9ab-42da-b395-dfb453a632a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a353c4943d18429a94aa9d93b67f04db)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:49.885144 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05113dd2-b9ab-42da-b395-dfb453a632a5 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:49.976598 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf383e8-f74a-4db9-ba57-6614d36a7982 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:49.985808 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505af209-dbde-4c3f-9218-e5301cb2ab21 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17da357e555420799ee681bffbcc268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:49.986736 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505af209-dbde-4c3f-9218-e5301cb2ab21 service nova] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:49.987207 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505af209-dbde-4c3f-9218-e5301cb2ab21 service nova] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:49.987622 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505af209-dbde-4c3f-9218-e5301cb2ab21 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:49.999413 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505af209-dbde-4c3f-9218-e5301cb2ab21 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:50.001724 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505af209-dbde-4c3f-9218-e5301cb2ab21 service nova] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:50.029633 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 855/1723] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:55:49 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:55:50.077550 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:50.090257 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505af209-dbde-4c3f-9218-e5301cb2ab21 service nova] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:50.092028 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-505af209-dbde-4c3f-9218-e5301cb2ab21 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:50.112272 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-05113dd2-b9ab-42da-b395-dfb453a632a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a353c4943d18429a94aa9d93b67f04db', 'request_id': 'req-05113dd2-b9ab-42da-b395-dfb453a632a5'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:50.127621 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-05113dd2-b9ab-42da-b395-dfb453a632a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a353c4943d18429a94aa9d93b67f04db', 'request_id': 'req-05113dd2-b9ab-42da-b395-dfb453a632a5'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:50.265253 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-05113dd2-b9ab-42da-b395-dfb453a632a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a353c4943d18429a94aa9d93b67f04db', 'request_id': 'req-05113dd2-b9ab-42da-b395-dfb453a632a5'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:50.293553 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-05113dd2-b9ab-42da-b395-dfb453a632a5 admin admin] The token cache is being invalidate because project a353c4943d18429a94aa9d93b67f04db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:55:50.298119 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-05113dd2-b9ab-42da-b395-dfb453a632a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:50.299108 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-05113dd2-b9ab-42da-b395-dfb453a632a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:50.305136 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 869/1724] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 08:55:49 2022] DELETE /identity/v3/projects/a353c4943d18429a94aa9d93b67f04db => generated 0 bytes in 657 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:55:50.306402 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05113dd2-b9ab-42da-b395-dfb453a632a5 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:50.312714 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-05113dd2-b9ab-42da-b395-dfb453a632a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:50.391108 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505af209-dbde-4c3f-9218-e5301cb2ab21 service nova] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:50.476401 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05113dd2-b9ab-42da-b395-dfb453a632a5 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:50.491438 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9f62b0-a003-455f-acb8-bad4d0a935d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:50.492455 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9f62b0-a003-455f-acb8-bad4d0a935d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:50.492840 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9f62b0-a003-455f-acb8-bad4d0a935d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:50.493775 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9f62b0-a003-455f-acb8-bad4d0a935d7 admin admin] PATH_INFO: `/v3/projects/c4169dc29f4f4c1fbbb88cdd84bcd649` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:50.498244 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9f62b0-a003-455f-acb8-bad4d0a935d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4169dc29f4f4c1fbbb88cdd84bcd649)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:50.500830 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9f62b0-a003-455f-acb8-bad4d0a935d7 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:50.559716 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff29423-8afd-464b-a6b5-e548a8dd0f5a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0ebb5ccabef45378f0c1da69721e051', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:50.560502 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff29423-8afd-464b-a6b5-e548a8dd0f5a service swift] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:50.566200 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff29423-8afd-464b-a6b5-e548a8dd0f5a service swift] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:50.566547 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff29423-8afd-464b-a6b5-e548a8dd0f5a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:50.571795 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-aff29423-8afd-464b-a6b5-e548a8dd0f5a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:50.749420 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff29423-8afd-464b-a6b5-e548a8dd0f5a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:50.751372 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff29423-8afd-464b-a6b5-e548a8dd0f5a service swift] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:50.775090 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 856/1725] 10.222.0.55 () {60 vars in 1384 bytes} [Wed Aug 24 08:55:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 685 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 08:55:50.775720 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ea1f09-b37e-488e-8928-8f8d936cb967 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:50.776012 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ea1f09-b37e-488e-8928-8f8d936cb967 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:50.776286 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ea1f09-b37e-488e-8928-8f8d936cb967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:50.860153 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-40ea1f09-b37e-488e-8928-8f8d936cb967 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:50.872665 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-40ea1f09-b37e-488e-8928-8f8d936cb967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:50.891204 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-4f9f62b0-a003-455f-acb8-bad4d0a935d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4169dc29f4f4c1fbbb88cdd84bcd649', 'request_id': 'req-4f9f62b0-a003-455f-acb8-bad4d0a935d7'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:50.922627 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-4f9f62b0-a003-455f-acb8-bad4d0a935d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4169dc29f4f4c1fbbb88cdd84bcd649', 'request_id': 'req-4f9f62b0-a003-455f-acb8-bad4d0a935d7'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:51.017104 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-4f9f62b0-a003-455f-acb8-bad4d0a935d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4169dc29f4f4c1fbbb88cdd84bcd649', 'request_id': 'req-4f9f62b0-a003-455f-acb8-bad4d0a935d7'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:51.052105 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-4f9f62b0-a003-455f-acb8-bad4d0a935d7 admin admin] The token cache is being invalidate because project c4169dc29f4f4c1fbbb88cdd84bcd649 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:55:51.058684 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-4f9f62b0-a003-455f-acb8-bad4d0a935d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:51.058684 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-4f9f62b0-a003-455f-acb8-bad4d0a935d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:51.068704 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 870/1726] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 08:55:50 2022] DELETE /identity/v3/projects/c4169dc29f4f4c1fbbb88cdd84bcd649 => generated 0 bytes in 762 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:55:51.074062 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca58a1e-7877-4f04-9f93-b1c3f4f07c45 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:51.074639 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca58a1e-7877-4f04-9f93-b1c3f4f07c45 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:51.075059 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca58a1e-7877-4f04-9f93-b1c3f4f07c45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:51.150801 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-1ca58a1e-7877-4f04-9f93-b1c3f4f07c45 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:51.160079 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 857/1727] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:55:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:51.164624 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-1ca58a1e-7877-4f04-9f93-b1c3f4f07c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:51.165443 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93caaebe-6922-4ce2-8f54-168f7ad34aac None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:51.165591 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93caaebe-6922-4ce2-8f54-168f7ad34aac None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:51.165874 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93caaebe-6922-4ce2-8f54-168f7ad34aac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:51.218798 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 871/1728] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:55:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:51.231241 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-93caaebe-6922-4ce2-8f54-168f7ad34aac None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:51.231889 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02794ffe-aa52-49c9-a53c-2023aed38e79 None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:51.232608 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02794ffe-aa52-49c9-a53c-2023aed38e79 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:51.233048 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-93caaebe-6922-4ce2-8f54-168f7ad34aac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:51.236004 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02794ffe-aa52-49c9-a53c-2023aed38e79 None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:51.244066 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 872/1729] 10.222.0.55 () {54 vars in 912 bytes} [Wed Aug 24 08:55:51 2022] GET /identity => generated 270 bytes in 24 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:55:51.250158 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c411ba4-4bde-4ae7-a68a-0f6438d94b86 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:51.251253 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c411ba4-4bde-4ae7-a68a-0f6438d94b86 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:51.251846 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c411ba4-4bde-4ae7-a68a-0f6438d94b86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:51.256952 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 858/1730] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:55:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:51.256952 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b91d03-6d4b-45a8-81ae-cc51d99dc494 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:51.256952 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b91d03-6d4b-45a8-81ae-cc51d99dc494 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:51.256952 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b91d03-6d4b-45a8-81ae-cc51d99dc494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:51.298607 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-b0b91d03-6d4b-45a8-81ae-cc51d99dc494 None None] MFA Rules not processed for user `28b58388d1664510b0b507323bca5ae8`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:51.305328 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-b0b91d03-6d4b-45a8-81ae-cc51d99dc494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:51.365661 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-1c411ba4-4bde-4ae7-a68a-0f6438d94b86 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:51.381427 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-1c411ba4-4bde-4ae7-a68a-0f6438d94b86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:51.434998 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 873/1731] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:55:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:51.448951 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 859/1732] 10.222.0.55 () {58 vars in 1009 bytes} [Wed Aug 24 08:55:51 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:51.450232 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b91d03-6d4b-45a8-81ae-cc51d99dc494 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:51.452527 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c411ba4-4bde-4ae7-a68a-0f6438d94b86 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:51.476250 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b91d03-6d4b-45a8-81ae-cc51d99dc494 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:51.477316 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c411ba4-4bde-4ae7-a68a-0f6438d94b86 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:51.503214 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085432e2-ef53-4aaa-b2c8-e516075abe42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:51.504077 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839e8ab4-5b3f-475c-a67e-94cd796c2123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:51.506126 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839e8ab4-5b3f-475c-a67e-94cd796c2123 admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:51.507204 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085432e2-ef53-4aaa-b2c8-e516075abe42 admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:51.507880 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839e8ab4-5b3f-475c-a67e-94cd796c2123 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:51.508268 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085432e2-ef53-4aaa-b2c8-e516075abe42 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:51.508963 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839e8ab4-5b3f-475c-a67e-94cd796c2123 admin admin] PATH_INFO: `/v3/domains` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:51.509371 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085432e2-ef53-4aaa-b2c8-e516075abe42 admin admin] PATH_INFO: `/v3/domains` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:51.510319 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085432e2-ef53-4aaa-b2c8-e516075abe42 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:55:51.512859 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839e8ab4-5b3f-475c-a67e-94cd796c2123 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:55:51.517053 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839e8ab4-5b3f-475c-a67e-94cd796c2123 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:51.517650 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085432e2-ef53-4aaa-b2c8-e516075abe42 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:51.519547 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839e8ab4-5b3f-475c-a67e-94cd796c2123 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:51.521481 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085432e2-ef53-4aaa-b2c8-e516075abe42 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:51.548267 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 860/1733] 10.222.0.55 () {58 vars in 1153 bytes} [Wed Aug 24 08:55:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:55:51.549370 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839e8ab4-5b3f-475c-a67e-94cd796c2123 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:51.551207 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-839e8ab4-5b3f-475c-a67e-94cd796c2123 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:51.554617 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 874/1734] 10.222.0.55 () {58 vars in 1153 bytes} [Wed Aug 24 08:55:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:55:51.555578 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085432e2-ef53-4aaa-b2c8-e516075abe42 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:51.579073 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085432e2-ef53-4aaa-b2c8-e516075abe42 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:51.584098 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839e8ab4-5b3f-475c-a67e-94cd796c2123 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:51.592093 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08a53ce-47b2-4974-a4e1-073f2a865193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:51.593908 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08a53ce-47b2-4974-a4e1-073f2a865193 admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:51.593908 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08a53ce-47b2-4974-a4e1-073f2a865193 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:51.593908 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08a53ce-47b2-4974-a4e1-073f2a865193 admin admin] PATH_INFO: `/v3/projects` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:51.595777 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7ece6a-16f8-4b55-88fd-5bec21d132d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:51.596777 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ece6a-16f8-4b55-88fd-5bec21d132d3 service neutron] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:51.597428 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08a53ce-47b2-4974-a4e1-073f2a865193 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:51.599780 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ece6a-16f8-4b55-88fd-5bec21d132d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:51.600247 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ece6a-16f8-4b55-88fd-5bec21d132d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:51.602083 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08a53ce-47b2-4974-a4e1-073f2a865193 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:51.616073 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7ece6a-16f8-4b55-88fd-5bec21d132d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:51.619149 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7ece6a-16f8-4b55-88fd-5bec21d132d3 service neutron] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:51.637557 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 861/1735] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:55:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:55:51.640121 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7ece6a-16f8-4b55-88fd-5bec21d132d3 service neutron] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:51.655355 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7ece6a-16f8-4b55-88fd-5bec21d132d3 service neutron] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:51.663108 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c38ffb-8e75-48ff-bddb-e0b5397c497a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:51.663890 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c38ffb-8e75-48ff-bddb-e0b5397c497a admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:51.664293 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c38ffb-8e75-48ff-bddb-e0b5397c497a admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:51.664715 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c38ffb-8e75-48ff-bddb-e0b5397c497a admin admin] PATH_INFO: `/v3/projects` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:51.665638 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c38ffb-8e75-48ff-bddb-e0b5397c497a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:51.675428 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c38ffb-8e75-48ff-bddb-e0b5397c497a admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:51.746469 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 875/1736] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 08:55:51 2022] POST /identity/v3/projects => generated 389 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:55:51.753102 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08a53ce-47b2-4974-a4e1-073f2a865193 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:51.826867 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 862/1737] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 08:55:51 2022] POST /identity/v3/projects => generated 387 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:55:51.831537 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c38ffb-8e75-48ff-bddb-e0b5397c497a admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:51.946400 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08a53ce-47b2-4974-a4e1-073f2a865193 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:51.985936 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:52.057184 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c38ffb-8e75-48ff-bddb-e0b5397c497a admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:52.074712 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd18284b-4951-45ef-9b4f-96c5b44ab13a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:52.075613 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd18284b-4951-45ef-9b4f-96c5b44ab13a admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:52.076000 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd18284b-4951-45ef-9b4f-96c5b44ab13a admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:52.076597 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd18284b-4951-45ef-9b4f-96c5b44ab13a admin admin] PATH_INFO: `/v3/users` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:52.078382 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd18284b-4951-45ef-9b4f-96c5b44ab13a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:52.092464 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd18284b-4951-45ef-9b4f-96c5b44ab13a admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:52.238999 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a98406-2cf2-4855-ab54-107e1dd05fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:52.242445 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a98406-2cf2-4855-ab54-107e1dd05fc1 admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:52.243407 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a98406-2cf2-4855-ab54-107e1dd05fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:52.244858 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a98406-2cf2-4855-ab54-107e1dd05fc1 admin admin] PATH_INFO: `/v3/users` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:52.247065 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a98406-2cf2-4855-ab54-107e1dd05fc1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:52.253889 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a98406-2cf2-4855-ab54-107e1dd05fc1 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:52.273366 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 863/1738] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 08:55:51 2022] POST /identity/v3/users => generated 294 bytes in 442 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:55:52.275202 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df4dd13-e5cb-476e-b18f-76adeb8eafc7 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:52.275494 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df4dd13-e5cb-476e-b18f-76adeb8eafc7 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:52.275840 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df4dd13-e5cb-476e-b18f-76adeb8eafc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:52.354493 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:52.374515 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-4df4dd13-e5cb-476e-b18f-76adeb8eafc7 None None] MFA Rules not processed for user `f5d7aee76519492e81b0048edebba36f`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:52.374515 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-4df4dd13-e5cb-476e-b18f-76adeb8eafc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:52.415006 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 876/1739] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 08:55:51 2022] POST /identity/v3/users => generated 295 bytes in 662 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:55:52.416622 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a98406-2cf2-4855-ab54-107e1dd05fc1 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:52.438478 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a98406-2cf2-4855-ab54-107e1dd05fc1 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:52.470265 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c399f157-86d5-4c8e-95f0-fcfc77f3d79d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:52.471329 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c399f157-86d5-4c8e-95f0-fcfc77f3d79d admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:52.471763 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c399f157-86d5-4c8e-95f0-fcfc77f3d79d admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:52.472225 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c399f157-86d5-4c8e-95f0-fcfc77f3d79d admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:52.473311 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c399f157-86d5-4c8e-95f0-fcfc77f3d79d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:52.476175 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c399f157-86d5-4c8e-95f0-fcfc77f3d79d admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:52.535330 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 877/1740] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:55:52 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:55:52.537305 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c399f157-86d5-4c8e-95f0-fcfc77f3d79d admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:52.581398 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c399f157-86d5-4c8e-95f0-fcfc77f3d79d admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:52.623050 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fec6dab-41ed-4eb1-9efe-df960a522d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:52.624276 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fec6dab-41ed-4eb1-9efe-df960a522d28 admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:52.624276 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fec6dab-41ed-4eb1-9efe-df960a522d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:52.624276 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fec6dab-41ed-4eb1-9efe-df960a522d28 admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:52.628924 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fec6dab-41ed-4eb1-9efe-df960a522d28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:52.628924 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fec6dab-41ed-4eb1-9efe-df960a522d28 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:52.671876 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 878/1741] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:55:52 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:55:52.673120 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fec6dab-41ed-4eb1-9efe-df960a522d28 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:52.678375 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 864/1742] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:55:52 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 404 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:52.696821 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df4dd13-e5cb-476e-b18f-76adeb8eafc7 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:52.722528 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fec6dab-41ed-4eb1-9efe-df960a522d28 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:52.740807 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8a7f7d-2c96-4ab6-ba7c-ecde940f3906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:52.742990 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a7f7d-2c96-4ab6-ba7c-ecde940f3906 admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:52.742990 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a7f7d-2c96-4ab6-ba7c-ecde940f3906 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:52.742990 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a7f7d-2c96-4ab6-ba7c-ecde940f3906 admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:52.742990 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8a7f7d-2c96-4ab6-ba7c-ecde940f3906 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:52.746489 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8a7f7d-2c96-4ab6-ba7c-ecde940f3906 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:52.762910 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df4dd13-e5cb-476e-b18f-76adeb8eafc7 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:52.785145 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52cdf77-b8db-474e-8728-3041adebbbe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:52.786615 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52cdf77-b8db-474e-8728-3041adebbbe9 admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:52.787990 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52cdf77-b8db-474e-8728-3041adebbbe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:52.788281 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52cdf77-b8db-474e-8728-3041adebbbe9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:52.788997 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52cdf77-b8db-474e-8728-3041adebbbe9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:52.794311 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 879/1743] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:55:52 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:55:52.794856 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8a7f7d-2c96-4ab6-ba7c-ecde940f3906 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:52.804500 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:52.813385 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52cdf77-b8db-474e-8728-3041adebbbe9 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:52.822882 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8a7f7d-2c96-4ab6-ba7c-ecde940f3906 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:52.836169 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 865/1744] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:55:52 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:55:52.842013 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12a4e16-38ed-43f3-8822-509d6d90fadd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:52.846660 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12a4e16-38ed-43f3-8822-509d6d90fadd admin admin] REQUEST_METHOD: `PUT` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:52.846960 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12a4e16-38ed-43f3-8822-509d6d90fadd admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:52.847250 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12a4e16-38ed-43f3-8822-509d6d90fadd admin admin] PATH_INFO: `/v3/projects/b343199620ed40ccae7a5c4772495050/users/d62f2e0a8473411da8dc0ae3f473fe87/roles/b674fd0a592e4e0cada811bba5ceea4d` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:52.847795 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52cdf77-b8db-474e-8728-3041adebbbe9 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:52.866254 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52cdf77-b8db-474e-8728-3041adebbbe9 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:52.895351 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aac5674-7cd7-4de7-a8c4-add253eebef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:52.901912 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aac5674-7cd7-4de7-a8c4-add253eebef1 admin admin] REQUEST_METHOD: `PUT` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:52.901912 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aac5674-7cd7-4de7-a8c4-add253eebef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:52.901912 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aac5674-7cd7-4de7-a8c4-add253eebef1 admin admin] PATH_INFO: `/v3/projects/cd257063cc214c52b3aad9c6c807999e/users/a13f2f875de140c6ad1ee6a8e681c3d4/roles/b674fd0a592e4e0cada811bba5ceea4d` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:52.925449 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12a4e16-38ed-43f3-8822-509d6d90fadd admin admin] RBAC: Authorizing `identity:create_grant(project_id=b343199620ed40ccae7a5c4772495050, user_id=d62f2e0a8473411da8dc0ae3f473fe87, role_id=b674fd0a592e4e0cada811bba5ceea4d)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:52.939959 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12a4e16-38ed-43f3-8822-509d6d90fadd admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:52.965530 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aac5674-7cd7-4de7-a8c4-add253eebef1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd257063cc214c52b3aad9c6c807999e, user_id=a13f2f875de140c6ad1ee6a8e681c3d4, role_id=b674fd0a592e4e0cada811bba5ceea4d)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:52.969717 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aac5674-7cd7-4de7-a8c4-add253eebef1 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:53.040106 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 880/1745] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 08:55:52 2022] PUT /identity/v3/projects/b343199620ed40ccae7a5c4772495050/users/d62f2e0a8473411da8dc0ae3f473fe87/roles/b674fd0a592e4e0cada811bba5ceea4d => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:55:53.043939 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a49eda-1ef9-4ad0-bd23-6ca91acb52ef None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:53.045772 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a49eda-1ef9-4ad0-bd23-6ca91acb52ef None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:53.046066 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a49eda-1ef9-4ad0-bd23-6ca91acb52ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:53.124910 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 866/1746] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 08:55:52 2022] PUT /identity/v3/projects/cd257063cc214c52b3aad9c6c807999e/users/a13f2f875de140c6ad1ee6a8e681c3d4/roles/b674fd0a592e4e0cada811bba5ceea4d => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:55:53.128881 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b538c5ae-c9c5-4132-8cc4-e9e45a6e0282 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:53.129327 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b538c5ae-c9c5-4132-8cc4-e9e45a6e0282 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:53.129708 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b538c5ae-c9c5-4132-8cc4-e9e45a6e0282 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:53.148681 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-33a49eda-1ef9-4ad0-bd23-6ca91acb52ef None None] MFA Rules not processed for user `d62f2e0a8473411da8dc0ae3f473fe87`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:53.149989 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-33a49eda-1ef9-4ad0-bd23-6ca91acb52ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:53.199801 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-b538c5ae-c9c5-4132-8cc4-e9e45a6e0282 None None] MFA Rules not processed for user `a13f2f875de140c6ad1ee6a8e681c3d4`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:53.201474 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-b538c5ae-c9c5-4132-8cc4-e9e45a6e0282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:53.467061 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 867/1747] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:55:53 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 339 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:53.470456 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01845e46-32fd-4168-b81d-c559d48e1b4f None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:53.470456 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01845e46-32fd-4168-b81d-c559d48e1b4f None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:53.470456 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01845e46-32fd-4168-b81d-c559d48e1b4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:53.474641 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:53.517265 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-01845e46-32fd-4168-b81d-c559d48e1b4f None None] MFA Rules not processed for user `f56524e822e641bc9bb11eddfe27f85d`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:53.517265 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-01845e46-32fd-4168-b81d-c559d48e1b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:53.658683 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 881/1748] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:55:53 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 616 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:55:53.660352 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a49eda-1ef9-4ad0-bd23-6ca91acb52ef None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:53.807950 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 868/1749] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:55:53 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:53.809599 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01845e46-32fd-4168-b81d-c559d48e1b4f None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:53.965336 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a49eda-1ef9-4ad0-bd23-6ca91acb52ef None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:53.995549 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:DELETE=3,keystone:SELECT=92,keystone:INSERT=5 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:54.064331 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01845e46-32fd-4168-b81d-c559d48e1b4f None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:54.074077 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9df3c2-1ccb-4fb2-a450-10ff387a48e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:54.075159 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9df3c2-1ccb-4fb2-a450-10ff387a48e6 admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:54.075671 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9df3c2-1ccb-4fb2-a450-10ff387a48e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:54.076116 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9df3c2-1ccb-4fb2-a450-10ff387a48e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:54.077637 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9df3c2-1ccb-4fb2-a450-10ff387a48e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:54.089468 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9df3c2-1ccb-4fb2-a450-10ff387a48e6 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:54.090762 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc1e935-0aa0-4c10-8dd1-21baa980068a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:54.091481 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc1e935-0aa0-4c10-8dd1-21baa980068a admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:54.092528 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc1e935-0aa0-4c10-8dd1-21baa980068a admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:54.092866 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc1e935-0aa0-4c10-8dd1-21baa980068a admin admin] PATH_INFO: `/v3/projects` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:54.093692 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc1e935-0aa0-4c10-8dd1-21baa980068a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:54.097094 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc1e935-0aa0-4c10-8dd1-21baa980068a admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:54.245886 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 869/1750] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 08:55:53 2022] POST /identity/v3/projects => generated 385 bytes in 437 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:55:54.252865 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc1e935-0aa0-4c10-8dd1-21baa980068a admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:54.336280 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 882/1751] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 08:55:53 2022] POST /identity/v3/projects => generated 389 bytes in 677 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:55:54.337885 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9df3c2-1ccb-4fb2-a450-10ff387a48e6 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:54.520018 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc1e935-0aa0-4c10-8dd1-21baa980068a admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:54.658822 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9df3c2-1ccb-4fb2-a450-10ff387a48e6 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:54.681938 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f636ac2-63d3-47ac-ab24-d26ebfb0b220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:54.682983 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f636ac2-63d3-47ac-ab24-d26ebfb0b220 admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:54.683472 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f636ac2-63d3-47ac-ab24-d26ebfb0b220 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:54.684153 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f636ac2-63d3-47ac-ab24-d26ebfb0b220 admin admin] PATH_INFO: `/v3/users` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:54.685346 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f636ac2-63d3-47ac-ab24-d26ebfb0b220 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:54.693211 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f636ac2-63d3-47ac-ab24-d26ebfb0b220 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:54.780340 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b64fc20-2a84-410b-bee5-91adf6c7519a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:54.783580 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b64fc20-2a84-410b-bee5-91adf6c7519a admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:54.786165 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b64fc20-2a84-410b-bee5-91adf6c7519a admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:54.787133 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b64fc20-2a84-410b-bee5-91adf6c7519a admin admin] PATH_INFO: `/v3/users` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:54.788237 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b64fc20-2a84-410b-bee5-91adf6c7519a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:54.798500 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b64fc20-2a84-410b-bee5-91adf6c7519a admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:54.870210 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 883/1752] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 08:55:54 2022] POST /identity/v3/users => generated 301 bytes in 529 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:55:54.873353 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f636ac2-63d3-47ac-ab24-d26ebfb0b220 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:54.928159 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 870/1753] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 08:55:54 2022] POST /identity/v3/users => generated 299 bytes in 677 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:55:54.934963 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b64fc20-2a84-410b-bee5-91adf6c7519a admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:54.951808 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f636ac2-63d3-47ac-ab24-d26ebfb0b220 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:54.979527 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f290326-355f-4b9d-86c9-44ec6b106018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:54.980674 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f290326-355f-4b9d-86c9-44ec6b106018 admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:54.981299 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f290326-355f-4b9d-86c9-44ec6b106018 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:54.981418 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b64fc20-2a84-410b-bee5-91adf6c7519a admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:54.982085 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f290326-355f-4b9d-86c9-44ec6b106018 admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:54.983475 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f290326-355f-4b9d-86c9-44ec6b106018 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:54.991470 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f290326-355f-4b9d-86c9-44ec6b106018 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:54.997384 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:55.021435 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c87e6b2-a76d-4f1e-9e04-3382eabc3a47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:55.022109 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c87e6b2-a76d-4f1e-9e04-3382eabc3a47 admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:55.022423 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c87e6b2-a76d-4f1e-9e04-3382eabc3a47 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:55.022698 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c87e6b2-a76d-4f1e-9e04-3382eabc3a47 admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:55.023380 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c87e6b2-a76d-4f1e-9e04-3382eabc3a47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:55.025103 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c87e6b2-a76d-4f1e-9e04-3382eabc3a47 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:55.051241 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 884/1754] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:55:54 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:55:55.058822 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f290326-355f-4b9d-86c9-44ec6b106018 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:55.070655 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 871/1755] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:55:54 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:55:55.077053 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c87e6b2-a76d-4f1e-9e04-3382eabc3a47 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:55.103629 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c87e6b2-a76d-4f1e-9e04-3382eabc3a47 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:55.115717 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab664de9-44a3-4937-a924-ca65d6aefe93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:55.116416 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab664de9-44a3-4937-a924-ca65d6aefe93 admin admin] REQUEST_METHOD: `PUT` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:55.116786 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab664de9-44a3-4937-a924-ca65d6aefe93 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:55.117062 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab664de9-44a3-4937-a924-ca65d6aefe93 admin admin] PATH_INFO: `/v3/projects/4c613726d528442e88e16f68524a1bac/users/c7fd6ba91f854876bb9144cdc0cd0c4b/roles/b870cd1acaa744e1ae1d58b9ac283821` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:55.122033 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f290326-355f-4b9d-86c9-44ec6b106018 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:55.134339 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b4ab72-c475-4af3-aebf-d0401ce3a84b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:55.136774 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b4ab72-c475-4af3-aebf-d0401ce3a84b admin admin] REQUEST_METHOD: `PUT` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:55.137851 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b4ab72-c475-4af3-aebf-d0401ce3a84b admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:55.139810 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b4ab72-c475-4af3-aebf-d0401ce3a84b admin admin] PATH_INFO: `/v3/projects/b1c74c0a8a0d497083e9024df2e1fef1/users/59045b09206140e1b3be905f654b1dd9/roles/b870cd1acaa744e1ae1d58b9ac283821` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:55.153239 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab664de9-44a3-4937-a924-ca65d6aefe93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c613726d528442e88e16f68524a1bac, user_id=c7fd6ba91f854876bb9144cdc0cd0c4b, role_id=b870cd1acaa744e1ae1d58b9ac283821)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:55.157182 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab664de9-44a3-4937-a924-ca65d6aefe93 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:55.234891 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b4ab72-c475-4af3-aebf-d0401ce3a84b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1c74c0a8a0d497083e9024df2e1fef1, user_id=59045b09206140e1b3be905f654b1dd9, role_id=b870cd1acaa744e1ae1d58b9ac283821)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:55.236479 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b4ab72-c475-4af3-aebf-d0401ce3a84b admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:55.340035 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 872/1756] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 08:55:55 2022] PUT /identity/v3/projects/4c613726d528442e88e16f68524a1bac/users/c7fd6ba91f854876bb9144cdc0cd0c4b/roles/b870cd1acaa744e1ae1d58b9ac283821 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:55:55.342983 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab664de9-44a3-4937-a924-ca65d6aefe93 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:55.369054 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 885/1757] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 08:55:55 2022] PUT /identity/v3/projects/b1c74c0a8a0d497083e9024df2e1fef1/users/59045b09206140e1b3be905f654b1dd9/roles/b870cd1acaa744e1ae1d58b9ac283821 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:55:55.377268 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b4ab72-c475-4af3-aebf-d0401ce3a84b admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:55.588120 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab664de9-44a3-4937-a924-ca65d6aefe93 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:55.749091 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc4ba17-ae2b-4275-8238-e6adbe4b93fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:55.750735 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc4ba17-ae2b-4275-8238-e6adbe4b93fd admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:55.751184 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc4ba17-ae2b-4275-8238-e6adbe4b93fd admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:55.751590 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc4ba17-ae2b-4275-8238-e6adbe4b93fd admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:55.752921 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc4ba17-ae2b-4275-8238-e6adbe4b93fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:55.757390 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc4ba17-ae2b-4275-8238-e6adbe4b93fd admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:55.807603 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 873/1758] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:55:55 2022] GET /identity/v3/roles => generated 1420 bytes in 467 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:55:55.818222 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc4ba17-ae2b-4275-8238-e6adbe4b93fd admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:55.848001 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b4ab72-c475-4af3-aebf-d0401ce3a84b admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:55.850670 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc4ba17-ae2b-4275-8238-e6adbe4b93fd admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:55.865116 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c53442-3738-4946-a005-6ec07e603e19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:55.865490 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c53442-3738-4946-a005-6ec07e603e19 admin admin] REQUEST_METHOD: `PUT` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:55.866069 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c53442-3738-4946-a005-6ec07e603e19 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:55.866577 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c53442-3738-4946-a005-6ec07e603e19 admin admin] PATH_INFO: `/v3/projects/4c613726d528442e88e16f68524a1bac/users/c7fd6ba91f854876bb9144cdc0cd0c4b/roles/b870cd1acaa744e1ae1d58b9ac283821` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:55.867347 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e2328e-ee14-44ba-b995-8fd6db8be356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:55.881626 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e2328e-ee14-44ba-b995-8fd6db8be356 admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:55.882162 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e2328e-ee14-44ba-b995-8fd6db8be356 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:55.882649 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e2328e-ee14-44ba-b995-8fd6db8be356 admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:55.897579 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e2328e-ee14-44ba-b995-8fd6db8be356 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:55.902403 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c53442-3738-4946-a005-6ec07e603e19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c613726d528442e88e16f68524a1bac, user_id=c7fd6ba91f854876bb9144cdc0cd0c4b, role_id=b870cd1acaa744e1ae1d58b9ac283821)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:55.907971 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c53442-3738-4946-a005-6ec07e603e19 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:55.909140 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e2328e-ee14-44ba-b995-8fd6db8be356 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:55.957630 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 886/1759] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:55:55 2022] GET /identity/v3/roles => generated 1420 bytes in 581 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:55:55.961080 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec60a95-e344-4d65-81b0-50549ff656f5 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:55.965808 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec60a95-e344-4d65-81b0-50549ff656f5 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:55.965808 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec60a95-e344-4d65-81b0-50549ff656f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:55.981812 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 874/1760] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 08:55:55 2022] PUT /identity/v3/projects/4c613726d528442e88e16f68524a1bac/users/c7fd6ba91f854876bb9144cdc0cd0c4b/roles/b870cd1acaa744e1ae1d58b9ac283821 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:55:55.985070 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c53442-3738-4946-a005-6ec07e603e19 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:56.145976 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-9ec60a95-e344-4d65-81b0-50549ff656f5 None None] MFA Rules not processed for user `28b58388d1664510b0b507323bca5ae8`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:56.150108 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-9ec60a95-e344-4d65-81b0-50549ff656f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:56.262683 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:56.281402 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c53442-3738-4946-a005-6ec07e603e19 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:56.294750 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 887/1761] 10.222.0.59 () {58 vars in 1015 bytes} [Wed Aug 24 08:55:55 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:56.294750 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c02bf77-e175-478e-ae8b-cf1b3f5a316a None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:56.294750 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c02bf77-e175-478e-ae8b-cf1b3f5a316a None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:56.294750 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c02bf77-e175-478e-ae8b-cf1b3f5a316a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:56.308328 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb198504-a35d-4190-9ab9-da42b62f975c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:56.308328 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb198504-a35d-4190-9ab9-da42b62f975c admin admin] REQUEST_METHOD: `PUT` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:56.308328 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb198504-a35d-4190-9ab9-da42b62f975c admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:56.308328 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb198504-a35d-4190-9ab9-da42b62f975c admin admin] PATH_INFO: `/v3/projects/b1c74c0a8a0d497083e9024df2e1fef1/users/59045b09206140e1b3be905f654b1dd9/roles/b870cd1acaa744e1ae1d58b9ac283821` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:56.309197 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb198504-a35d-4190-9ab9-da42b62f975c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1c74c0a8a0d497083e9024df2e1fef1, user_id=59045b09206140e1b3be905f654b1dd9, role_id=b870cd1acaa744e1ae1d58b9ac283821)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:56.309197 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb198504-a35d-4190-9ab9-da42b62f975c admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:56.360156 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-7c02bf77-e175-478e-ae8b-cf1b3f5a316a None None] MFA Rules not processed for user `c7fd6ba91f854876bb9144cdc0cd0c4b`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:56.362123 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-7c02bf77-e175-478e-ae8b-cf1b3f5a316a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:56.372689 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:56.374062 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 875/1762] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 08:55:55 2022] PUT /identity/v3/projects/b1c74c0a8a0d497083e9024df2e1fef1/users/59045b09206140e1b3be905f654b1dd9/roles/b870cd1acaa744e1ae1d58b9ac283821 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:55:56.384876 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42d592d-d782-4976-96af-72104533ae00 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:56.385369 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42d592d-d782-4976-96af-72104533ae00 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:56.385852 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42d592d-d782-4976-96af-72104533ae00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:56.584082 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-f42d592d-d782-4976-96af-72104533ae00 None None] MFA Rules not processed for user `59045b09206140e1b3be905f654b1dd9`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:56.586337 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-f42d592d-d782-4976-96af-72104533ae00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:56.654477 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 888/1763] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:55:56 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 364 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:56.659625 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c02bf77-e175-478e-ae8b-cf1b3f5a316a None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:56.671248 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-7c02bf77-e175-478e-ae8b-cf1b3f5a316a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:56.822845 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 876/1764] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:55:56 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 435 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:56.822845 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42d592d-d782-4976-96af-72104533ae00 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:56.822845 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-f42d592d-d782-4976-96af-72104533ae00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:56.858509 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c02bf77-e175-478e-ae8b-cf1b3f5a316a None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:56.864000 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdd1ee4-ab52-4b62-9e7c-fb0496ee91f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:56.865188 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdd1ee4-ab52-4b62-9e7c-fb0496ee91f9 service neutron] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:56.865480 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdd1ee4-ab52-4b62-9e7c-fb0496ee91f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:56.865728 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdd1ee4-ab52-4b62-9e7c-fb0496ee91f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:56.883962 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdd1ee4-ab52-4b62-9e7c-fb0496ee91f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:56.885786 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdd1ee4-ab52-4b62-9e7c-fb0496ee91f9 service neutron] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:56.913365 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 889/1765] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:55:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:55:56.914586 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdd1ee4-ab52-4b62-9e7c-fb0496ee91f9 service neutron] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:56.920011 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-dcdd1ee4-ab52-4b62-9e7c-fb0496ee91f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:57.089004 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42d592d-d782-4976-96af-72104533ae00 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:57.101064 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8edd181-ec17-433a-a506-9292c6d60114 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbb9ac667f04c35a63b2689df18efed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:57.102114 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8edd181-ec17-433a-a506-9292c6d60114 service cinder] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:57.102611 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8edd181-ec17-433a-a506-9292c6d60114 service cinder] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:57.103100 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8edd181-ec17-433a-a506-9292c6d60114 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:57.121352 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8edd181-ec17-433a-a506-9292c6d60114 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:57.124356 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8edd181-ec17-433a-a506-9292c6d60114 service cinder] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:57.147893 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdd1ee4-ab52-4b62-9e7c-fb0496ee91f9 service neutron] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:57.156426 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 877/1766] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:55:56 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:55:57.173348 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90ff47a-1fe0-4749-a0aa-933eab066e51 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbb9ac667f04c35a63b2689df18efed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:57.174477 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90ff47a-1fe0-4749-a0aa-933eab066e51 service cinder] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:57.190655 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90ff47a-1fe0-4749-a0aa-933eab066e51 service cinder] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:57.191283 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90ff47a-1fe0-4749-a0aa-933eab066e51 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:57.253943 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90ff47a-1fe0-4749-a0aa-933eab066e51 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:57.254966 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8edd181-ec17-433a-a506-9292c6d60114 service cinder] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:57.267112 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90ff47a-1fe0-4749-a0aa-933eab066e51 service cinder] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:57.330085 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 890/1767] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:55:56 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:55:57.552042 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:57.586494 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8edd181-ec17-433a-a506-9292c6d60114 service cinder] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:57.600063 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0fd452-ec35-479f-a5bb-5fdfe082ec77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:57.602705 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0fd452-ec35-479f-a5bb-5fdfe082ec77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:57.603053 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0fd452-ec35-479f-a5bb-5fdfe082ec77 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:57.603341 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0fd452-ec35-479f-a5bb-5fdfe082ec77 admin admin] PATH_INFO: `/v3/users/d62f2e0a8473411da8dc0ae3f473fe87` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:57.606335 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0fd452-ec35-479f-a5bb-5fdfe082ec77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d62f2e0a8473411da8dc0ae3f473fe87)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:57.631334 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0fd452-ec35-479f-a5bb-5fdfe082ec77 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:57.705421 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d256272-b4eb-4a4b-b57e-e61f7a433e53 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:57.706443 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d256272-b4eb-4a4b-b57e-e61f7a433e53 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:57.707407 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d256272-b4eb-4a4b-b57e-e61f7a433e53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:57.803578 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:57.868337 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-0d256272-b4eb-4a4b-b57e-e61f7a433e53 None None] MFA Rules not processed for user `993697d33b7841668b1eabcdcd6b40f9`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:57.872914 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-0d256272-b4eb-4a4b-b57e-e61f7a433e53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:58.060612 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 891/1768] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:55:57 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 358 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:58.062336 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d256272-b4eb-4a4b-b57e-e61f7a433e53 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:58.095204 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d256272-b4eb-4a4b-b57e-e61f7a433e53 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:58.140691 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85377166-757e-4edf-bc65-d71ad0b90911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:58.148955 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85377166-757e-4edf-bc65-d71ad0b90911 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:58.149563 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85377166-757e-4edf-bc65-d71ad0b90911 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:58.151743 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85377166-757e-4edf-bc65-d71ad0b90911 admin admin] PATH_INFO: `/v3/users/a13f2f875de140c6ad1ee6a8e681c3d4` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:58.172922 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85377166-757e-4edf-bc65-d71ad0b90911 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a13f2f875de140c6ad1ee6a8e681c3d4)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:58.197079 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85377166-757e-4edf-bc65-d71ad0b90911 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:58.220509 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-5e0fd452-ec35-479f-a5bb-5fdfe082ec77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd62f2e0a8473411da8dc0ae3f473fe87'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:58.225788 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-5e0fd452-ec35-479f-a5bb-5fdfe082ec77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd62f2e0a8473411da8dc0ae3f473fe87'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:58.234128 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-5e0fd452-ec35-479f-a5bb-5fdfe082ec77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd62f2e0a8473411da8dc0ae3f473fe87'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:58.322644 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-5e0fd452-ec35-479f-a5bb-5fdfe082ec77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd62f2e0a8473411da8dc0ae3f473fe87'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:58.459781 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-5e0fd452-ec35-479f-a5bb-5fdfe082ec77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd62f2e0a8473411da8dc0ae3f473fe87'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:58.490258 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 878/1769] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 08:55:57 2022] DELETE /identity/v3/users/d62f2e0a8473411da8dc0ae3f473fe87 => generated 0 bytes in 1237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:55:58.502198 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b488ce2-51c4-47b7-9890-d4dd93c038c1 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:58.502802 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b488ce2-51c4-47b7-9890-d4dd93c038c1 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:58.503327 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b488ce2-51c4-47b7-9890-d4dd93c038c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:58.562278 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-1b488ce2-51c4-47b7-9890-d4dd93c038c1 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:58.564031 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-1b488ce2-51c4-47b7-9890-d4dd93c038c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:58.671272 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-85377166-757e-4edf-bc65-d71ad0b90911 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a13f2f875de140c6ad1ee6a8e681c3d4'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:58.673668 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-85377166-757e-4edf-bc65-d71ad0b90911 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a13f2f875de140c6ad1ee6a8e681c3d4'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:58.676789 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-85377166-757e-4edf-bc65-d71ad0b90911 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a13f2f875de140c6ad1ee6a8e681c3d4'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:58.688316 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 879/1770] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:55:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:58.742961 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b488ce2-51c4-47b7-9890-d4dd93c038c1 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:58.742961 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-1b488ce2-51c4-47b7-9890-d4dd93c038c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:58.812113 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-85377166-757e-4edf-bc65-d71ad0b90911 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a13f2f875de140c6ad1ee6a8e681c3d4'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:58.868964 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b488ce2-51c4-47b7-9890-d4dd93c038c1 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:58.878674 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358eea37-7345-438f-9309-f78309e6c448 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:58.879490 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358eea37-7345-438f-9309-f78309e6c448 service neutron] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:58.879870 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358eea37-7345-438f-9309-f78309e6c448 service neutron] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:58.880317 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358eea37-7345-438f-9309-f78309e6c448 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:58.905087 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358eea37-7345-438f-9309-f78309e6c448 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:58.906762 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358eea37-7345-438f-9309-f78309e6c448 service neutron] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:58.931669 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 880/1771] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:55:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:55:58.955339 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-85377166-757e-4edf-bc65-d71ad0b90911 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a13f2f875de140c6ad1ee6a8e681c3d4'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:59.029396 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 892/1772] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 08:55:58 2022] DELETE /identity/v3/users/a13f2f875de140c6ad1ee6a8e681c3d4 => generated 0 bytes in 968 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:55:59.035947 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915c63a3-b05f-4e46-b232-fcbff2440c6a None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:59.036350 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915c63a3-b05f-4e46-b232-fcbff2440c6a None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:59.036790 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915c63a3-b05f-4e46-b232-fcbff2440c6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:59.071468 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-915c63a3-b05f-4e46-b232-fcbff2440c6a None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:59.077616 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-915c63a3-b05f-4e46-b232-fcbff2440c6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:59.177340 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:59.311293 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 881/1773] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:55:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:59.338024 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85377166-757e-4edf-bc65-d71ad0b90911 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:59.340396 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-85377166-757e-4edf-bc65-d71ad0b90911 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:59.394598 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:59.577708 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85377166-757e-4edf-bc65-d71ad0b90911 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:59.591915 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbff4586-670a-405b-8cdc-8fb155bc4919 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:59.592455 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbff4586-670a-405b-8cdc-8fb155bc4919 service neutron] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:59.593499 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbff4586-670a-405b-8cdc-8fb155bc4919 service neutron] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:59.596262 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbff4586-670a-405b-8cdc-8fb155bc4919 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:59.632744 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbff4586-670a-405b-8cdc-8fb155bc4919 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:59.642805 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbff4586-670a-405b-8cdc-8fb155bc4919 service neutron] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:59.678032 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 893/1774] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:55:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:56:00.276586 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbc204a-7e6a-417d-a25e-d3dda9cb8cc7 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:00.277498 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbc204a-7e6a-417d-a25e-d3dda9cb8cc7 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:00.278032 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbc204a-7e6a-417d-a25e-d3dda9cb8cc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:00.416656 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-0dbc204a-7e6a-417d-a25e-d3dda9cb8cc7 None None] MFA Rules not processed for user `1105970982014891902ebbb9904a57a5`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:00.417937 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-0dbc204a-7e6a-417d-a25e-d3dda9cb8cc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:00.669964 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 882/1775] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:56:00 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 389 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:01.816784 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f5ef5b-3c34-4d48-8194-67b666ab35ae None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:01.817384 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f5ef5b-3c34-4d48-8194-67b666ab35ae None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:01.817897 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f5ef5b-3c34-4d48-8194-67b666ab35ae None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:01.819831 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 894/1776] 10.222.0.55 () {54 vars in 921 bytes} [Wed Aug 24 08:56:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:56:01.846580 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f66a26a-5423-4c36-8a12-4504444cdd8a None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:01.846580 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f66a26a-5423-4c36-8a12-4504444cdd8a None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:01.846580 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f66a26a-5423-4c36-8a12-4504444cdd8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:01.878464 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-0f66a26a-5423-4c36-8a12-4504444cdd8a None None] MFA Rules not processed for user `f17da357e555420799ee681bffbcc268`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:01.879721 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-0f66a26a-5423-4c36-8a12-4504444cdd8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:01.936487 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:02.065633 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 883/1777] 10.222.0.55 () {58 vars in 1018 bytes} [Wed Aug 24 08:56:01 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:02.314791 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f5ef5b-3c34-4d48-8194-67b666ab35ae None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:02.317484 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-74f5ef5b-3c34-4d48-8194-67b666ab35ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:02.349317 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f5ef5b-3c34-4d48-8194-67b666ab35ae None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:02.363290 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f66a26a-5423-4c36-8a12-4504444cdd8a None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:02.364958 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-0f66a26a-5423-4c36-8a12-4504444cdd8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:02.376932 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f15000-e18f-4313-9f94-67d44767df4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:02.381503 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f15000-e18f-4313-9f94-67d44767df4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:02.381503 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f15000-e18f-4313-9f94-67d44767df4c admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:02.386796 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f15000-e18f-4313-9f94-67d44767df4c admin admin] PATH_INFO: `/v3/projects/cd257063cc214c52b3aad9c6c807999e` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:02.388192 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f66a26a-5423-4c36-8a12-4504444cdd8a None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:02.391590 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f15000-e18f-4313-9f94-67d44767df4c admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd257063cc214c52b3aad9c6c807999e)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:02.393792 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f15000-e18f-4313-9f94-67d44767df4c admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:02.409755 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0197b834-3b8e-4c5d-8409-b9ecf1876ebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:02.410654 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0197b834-3b8e-4c5d-8409-b9ecf1876ebb admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:02.415846 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0197b834-3b8e-4c5d-8409-b9ecf1876ebb admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:02.415846 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0197b834-3b8e-4c5d-8409-b9ecf1876ebb admin admin] PATH_INFO: `/v3/projects/b343199620ed40ccae7a5c4772495050` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:02.423288 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0197b834-3b8e-4c5d-8409-b9ecf1876ebb admin admin] RBAC: Authorizing `identity:delete_project(project_id=b343199620ed40ccae7a5c4772495050)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:02.428306 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0197b834-3b8e-4c5d-8409-b9ecf1876ebb admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:02.614273 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-77f15000-e18f-4313-9f94-67d44767df4c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd257063cc214c52b3aad9c6c807999e', 'request_id': 'req-77f15000-e18f-4313-9f94-67d44767df4c'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:02.624042 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-0197b834-3b8e-4c5d-8409-b9ecf1876ebb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b343199620ed40ccae7a5c4772495050', 'request_id': 'req-0197b834-3b8e-4c5d-8409-b9ecf1876ebb'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:02.629990 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-77f15000-e18f-4313-9f94-67d44767df4c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd257063cc214c52b3aad9c6c807999e', 'request_id': 'req-77f15000-e18f-4313-9f94-67d44767df4c'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:02.731355 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-0197b834-3b8e-4c5d-8409-b9ecf1876ebb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b343199620ed40ccae7a5c4772495050', 'request_id': 'req-0197b834-3b8e-4c5d-8409-b9ecf1876ebb'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:02.746192 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-0197b834-3b8e-4c5d-8409-b9ecf1876ebb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b343199620ed40ccae7a5c4772495050', 'request_id': 'req-0197b834-3b8e-4c5d-8409-b9ecf1876ebb'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:02.771749 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-0197b834-3b8e-4c5d-8409-b9ecf1876ebb admin admin] The token cache is being invalidate because project b343199620ed40ccae7a5c4772495050 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:56:02.772409 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-0197b834-3b8e-4c5d-8409-b9ecf1876ebb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:02.781407 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-0197b834-3b8e-4c5d-8409-b9ecf1876ebb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:02.786815 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 884/1778] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 08:56:02 2022] DELETE /identity/v3/projects/b343199620ed40ccae7a5c4772495050 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:02.787794 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0197b834-3b8e-4c5d-8409-b9ecf1876ebb admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:02.789662 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-0197b834-3b8e-4c5d-8409-b9ecf1876ebb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:02.805764 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-77f15000-e18f-4313-9f94-67d44767df4c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd257063cc214c52b3aad9c6c807999e', 'request_id': 'req-77f15000-e18f-4313-9f94-67d44767df4c'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:02.851867 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-77f15000-e18f-4313-9f94-67d44767df4c admin admin] The token cache is being invalidate because project cd257063cc214c52b3aad9c6c807999e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:56:02.852597 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-77f15000-e18f-4313-9f94-67d44767df4c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:02.853026 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-77f15000-e18f-4313-9f94-67d44767df4c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:02.859925 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 895/1779] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 08:56:02 2022] DELETE /identity/v3/projects/cd257063cc214c52b3aad9c6c807999e => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:02.859925 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f15000-e18f-4313-9f94-67d44767df4c admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:02.859925 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-77f15000-e18f-4313-9f94-67d44767df4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:02.939184 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:03.018020 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0197b834-3b8e-4c5d-8409-b9ecf1876ebb admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:03.031554 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186f2a55-eea1-4513-a35e-a319c728bfc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:03.032916 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186f2a55-eea1-4513-a35e-a319c728bfc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:03.033431 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186f2a55-eea1-4513-a35e-a319c728bfc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:03.034073 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186f2a55-eea1-4513-a35e-a319c728bfc8 admin admin] PATH_INFO: `/v3/users/c7fd6ba91f854876bb9144cdc0cd0c4b` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:03.037684 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186f2a55-eea1-4513-a35e-a319c728bfc8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7fd6ba91f854876bb9144cdc0cd0c4b)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:03.042826 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186f2a55-eea1-4513-a35e-a319c728bfc8 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:03.079089 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f15000-e18f-4313-9f94-67d44767df4c admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:03.200874 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bbc3cd-eeee-48eb-9803-b3f9152f4e3c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17da357e555420799ee681bffbcc268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:03.201573 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bbc3cd-eeee-48eb-9803-b3f9152f4e3c service nova] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:03.201856 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bbc3cd-eeee-48eb-9803-b3f9152f4e3c service nova] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:03.202129 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bbc3cd-eeee-48eb-9803-b3f9152f4e3c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:03.204233 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-59bbc3cd-eeee-48eb-9803-b3f9152f4e3c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:03.229401 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bbc3cd-eeee-48eb-9803-b3f9152f4e3c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:03.231033 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bbc3cd-eeee-48eb-9803-b3f9152f4e3c service nova] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:03.246506 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 896/1780] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:56:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:56:03.248035 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bbc3cd-eeee-48eb-9803-b3f9152f4e3c service nova] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:03.250370 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-59bbc3cd-eeee-48eb-9803-b3f9152f4e3c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:03.275333 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bbc3cd-eeee-48eb-9803-b3f9152f4e3c service nova] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:03.284066 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d329607f-59d1-4788-89a9-54758daadcb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:03.285220 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d329607f-59d1-4788-89a9-54758daadcb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:03.285724 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d329607f-59d1-4788-89a9-54758daadcb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:03.286224 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d329607f-59d1-4788-89a9-54758daadcb1 admin admin] PATH_INFO: `/v3/users/59045b09206140e1b3be905f654b1dd9` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:03.289049 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d329607f-59d1-4788-89a9-54758daadcb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59045b09206140e1b3be905f654b1dd9)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:03.293029 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d329607f-59d1-4788-89a9-54758daadcb1 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:03.317118 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:03.352046 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-186f2a55-eea1-4513-a35e-a319c728bfc8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7fd6ba91f854876bb9144cdc0cd0c4b'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:03.355051 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-186f2a55-eea1-4513-a35e-a319c728bfc8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7fd6ba91f854876bb9144cdc0cd0c4b'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:03.357162 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-186f2a55-eea1-4513-a35e-a319c728bfc8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7fd6ba91f854876bb9144cdc0cd0c4b'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:03.425787 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-186f2a55-eea1-4513-a35e-a319c728bfc8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7fd6ba91f854876bb9144cdc0cd0c4b'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:03.571234 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-186f2a55-eea1-4513-a35e-a319c728bfc8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7fd6ba91f854876bb9144cdc0cd0c4b'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:03.594743 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 885/1781] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 08:56:02 2022] DELETE /identity/v3/users/c7fd6ba91f854876bb9144cdc0cd0c4b => generated 0 bytes in 807 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:03.639610 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-d329607f-59d1-4788-89a9-54758daadcb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59045b09206140e1b3be905f654b1dd9'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:03.641225 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-d329607f-59d1-4788-89a9-54758daadcb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59045b09206140e1b3be905f654b1dd9'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:03.643052 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-d329607f-59d1-4788-89a9-54758daadcb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59045b09206140e1b3be905f654b1dd9'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:03.715641 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-d329607f-59d1-4788-89a9-54758daadcb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59045b09206140e1b3be905f654b1dd9'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:03.812428 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-d329607f-59d1-4788-89a9-54758daadcb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59045b09206140e1b3be905f654b1dd9'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:03.836131 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 897/1782] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 08:56:03 2022] DELETE /identity/v3/users/59045b09206140e1b3be905f654b1dd9 => generated 0 bytes in 589 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:04.619558 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186f2a55-eea1-4513-a35e-a319c728bfc8 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:04.620969 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d329607f-59d1-4788-89a9-54758daadcb1 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:04.621974 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-186f2a55-eea1-4513-a35e-a319c728bfc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:04.623339 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-d329607f-59d1-4788-89a9-54758daadcb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:04.754349 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186f2a55-eea1-4513-a35e-a319c728bfc8 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:04.756621 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d329607f-59d1-4788-89a9-54758daadcb1 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:04.778252 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27e0e79-db7c-4efb-b06d-9db57fba1a30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:04.778252 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27e0e79-db7c-4efb-b06d-9db57fba1a30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:04.778252 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27e0e79-db7c-4efb-b06d-9db57fba1a30 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:04.778252 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27e0e79-db7c-4efb-b06d-9db57fba1a30 admin admin] PATH_INFO: `/v3/projects/4c613726d528442e88e16f68524a1bac` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:04.779220 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf447ac-96d8-418e-8516-c9e360cb0201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:04.779220 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf447ac-96d8-418e-8516-c9e360cb0201 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:04.779500 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27e0e79-db7c-4efb-b06d-9db57fba1a30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c613726d528442e88e16f68524a1bac)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:04.779657 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf447ac-96d8-418e-8516-c9e360cb0201 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:04.779657 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf447ac-96d8-418e-8516-c9e360cb0201 admin admin] PATH_INFO: `/v3/projects/b1c74c0a8a0d497083e9024df2e1fef1` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:04.784298 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27e0e79-db7c-4efb-b06d-9db57fba1a30 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:04.792617 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf447ac-96d8-418e-8516-c9e360cb0201 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1c74c0a8a0d497083e9024df2e1fef1)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:04.795550 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf447ac-96d8-418e-8516-c9e360cb0201 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:04.868588 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:04.974484 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-ddf447ac-96d8-418e-8516-c9e360cb0201 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1c74c0a8a0d497083e9024df2e1fef1', 'request_id': 'req-ddf447ac-96d8-418e-8516-c9e360cb0201'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:04.983697 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-ddf447ac-96d8-418e-8516-c9e360cb0201 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1c74c0a8a0d497083e9024df2e1fef1', 'request_id': 'req-ddf447ac-96d8-418e-8516-c9e360cb0201'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:05.002108 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-f27e0e79-db7c-4efb-b06d-9db57fba1a30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c613726d528442e88e16f68524a1bac', 'request_id': 'req-f27e0e79-db7c-4efb-b06d-9db57fba1a30'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:05.082917 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-ddf447ac-96d8-418e-8516-c9e360cb0201 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1c74c0a8a0d497083e9024df2e1fef1', 'request_id': 'req-ddf447ac-96d8-418e-8516-c9e360cb0201'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:05.097384 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-ddf447ac-96d8-418e-8516-c9e360cb0201 admin admin] The token cache is being invalidate because project b1c74c0a8a0d497083e9024df2e1fef1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:56:05.098270 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-ddf447ac-96d8-418e-8516-c9e360cb0201 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:05.098692 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-ddf447ac-96d8-418e-8516-c9e360cb0201 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:05.100736 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 898/1783] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 08:56:04 2022] DELETE /identity/v3/projects/b1c74c0a8a0d497083e9024df2e1fef1 => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:05.111336 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0423bf16-c6f7-40c6-9af2-da3663e616fe None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:05.111971 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0423bf16-c6f7-40c6-9af2-da3663e616fe None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:05.112653 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0423bf16-c6f7-40c6-9af2-da3663e616fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:05.156349 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-0423bf16-c6f7-40c6-9af2-da3663e616fe None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:05.159357 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-0423bf16-c6f7-40c6-9af2-da3663e616fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:05.240748 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-f27e0e79-db7c-4efb-b06d-9db57fba1a30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c613726d528442e88e16f68524a1bac', 'request_id': 'req-f27e0e79-db7c-4efb-b06d-9db57fba1a30'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:05.257767 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-f27e0e79-db7c-4efb-b06d-9db57fba1a30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c613726d528442e88e16f68524a1bac', 'request_id': 'req-f27e0e79-db7c-4efb-b06d-9db57fba1a30'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:05.315013 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-f27e0e79-db7c-4efb-b06d-9db57fba1a30 admin admin] The token cache is being invalidate because project 4c613726d528442e88e16f68524a1bac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:56:05.315013 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-f27e0e79-db7c-4efb-b06d-9db57fba1a30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:05.315013 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-f27e0e79-db7c-4efb-b06d-9db57fba1a30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:05.315013 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 886/1784] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 08:56:04 2022] DELETE /identity/v3/projects/4c613726d528442e88e16f68524a1bac => generated 0 bytes in 694 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:05.323268 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5781cc-00b5-4132-b7f8-edb070116ef4 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:05.324186 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5781cc-00b5-4132-b7f8-edb070116ef4 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:05.325149 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5781cc-00b5-4132-b7f8-edb070116ef4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:05.339634 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 899/1785] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:56:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:05.370216 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219c4d3f-0820-4249-ae2e-d2e5eb1f7742 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:05.370719 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219c4d3f-0820-4249-ae2e-d2e5eb1f7742 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:05.371260 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219c4d3f-0820-4249-ae2e-d2e5eb1f7742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:05.373158 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-8a5781cc-00b5-4132-b7f8-edb070116ef4 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:05.374525 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-8a5781cc-00b5-4132-b7f8-edb070116ef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:05.386231 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 887/1786] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:56:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:05.402809 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f27b03d-5390-44ef-b31e-c53f29f7e237 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:05.403661 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f27b03d-5390-44ef-b31e-c53f29f7e237 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:05.405026 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f27b03d-5390-44ef-b31e-c53f29f7e237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:05.418900 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-219c4d3f-0820-4249-ae2e-d2e5eb1f7742 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:05.421650 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-219c4d3f-0820-4249-ae2e-d2e5eb1f7742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:05.439460 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 900/1787] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:56:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:05.441102 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219c4d3f-0820-4249-ae2e-d2e5eb1f7742 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:05.460203 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219c4d3f-0820-4249-ae2e-d2e5eb1f7742 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:05.464006 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-2f27b03d-5390-44ef-b31e-c53f29f7e237 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:05.467484 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-2f27b03d-5390-44ef-b31e-c53f29f7e237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:05.469274 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2635e0aa-bb81-4797-9010-134c0d52f809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:05.471635 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2635e0aa-bb81-4797-9010-134c0d52f809 admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:05.472074 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2635e0aa-bb81-4797-9010-134c0d52f809 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:05.472419 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2635e0aa-bb81-4797-9010-134c0d52f809 admin admin] PATH_INFO: `/v3/domains` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:05.473779 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2635e0aa-bb81-4797-9010-134c0d52f809 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:56:05.479100 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2635e0aa-bb81-4797-9010-134c0d52f809 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:05.479100 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2635e0aa-bb81-4797-9010-134c0d52f809 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:05.494553 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 888/1788] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:56:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:05.496600 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f27b03d-5390-44ef-b31e-c53f29f7e237 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:05.506172 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 901/1789] 10.222.0.55 () {58 vars in 1153 bytes} [Wed Aug 24 08:56:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:56:05.507415 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2635e0aa-bb81-4797-9010-134c0d52f809 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:05.517895 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f27b03d-5390-44ef-b31e-c53f29f7e237 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:05.527106 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01a9fb0-e663-41a4-bc39-35094f213b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:05.553660 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01a9fb0-e663-41a4-bc39-35094f213b6f admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:05.553660 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01a9fb0-e663-41a4-bc39-35094f213b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:05.553660 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01a9fb0-e663-41a4-bc39-35094f213b6f admin admin] PATH_INFO: `/v3/domains` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:05.553660 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01a9fb0-e663-41a4-bc39-35094f213b6f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:56:05.553660 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01a9fb0-e663-41a4-bc39-35094f213b6f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:05.553660 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01a9fb0-e663-41a4-bc39-35094f213b6f admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:05.554824 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2635e0aa-bb81-4797-9010-134c0d52f809 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:05.554824 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c95cd82-b772-46a1-8b2f-f77368c94ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:05.554824 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c95cd82-b772-46a1-8b2f-f77368c94ea7 admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:05.554824 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c95cd82-b772-46a1-8b2f-f77368c94ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:05.555421 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c95cd82-b772-46a1-8b2f-f77368c94ea7 admin admin] PATH_INFO: `/v3/projects` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:05.555421 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c95cd82-b772-46a1-8b2f-f77368c94ea7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:05.555421 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c95cd82-b772-46a1-8b2f-f77368c94ea7 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:05.579167 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 889/1790] 10.222.0.55 () {58 vars in 1153 bytes} [Wed Aug 24 08:56:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:56:05.582230 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01a9fb0-e663-41a4-bc39-35094f213b6f admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:05.602085 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01a9fb0-e663-41a4-bc39-35094f213b6f admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:05.610635 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ba5855-8062-49e3-8aa0-a06979fb280a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:05.612230 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ba5855-8062-49e3-8aa0-a06979fb280a admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:05.613361 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ba5855-8062-49e3-8aa0-a06979fb280a admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:05.614262 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ba5855-8062-49e3-8aa0-a06979fb280a admin admin] PATH_INFO: `/v3/projects` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:05.616255 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ba5855-8062-49e3-8aa0-a06979fb280a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:05.620333 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ba5855-8062-49e3-8aa0-a06979fb280a admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:05.660033 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:05.683238 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 902/1791] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 08:56:05 2022] POST /identity/v3/projects => generated 409 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:56:05.688019 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c95cd82-b772-46a1-8b2f-f77368c94ea7 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:05.758941 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 890/1792] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 08:56:05 2022] POST /identity/v3/projects => generated 399 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:56:05.758941 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ba5855-8062-49e3-8aa0-a06979fb280a admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:05.877251 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c95cd82-b772-46a1-8b2f-f77368c94ea7 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:05.912484 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ba5855-8062-49e3-8aa0-a06979fb280a admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:05.924469 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89094351-d5c4-4ec0-b712-196b835e971e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:05.924469 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89094351-d5c4-4ec0-b712-196b835e971e admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:05.924469 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89094351-d5c4-4ec0-b712-196b835e971e admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:05.924469 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89094351-d5c4-4ec0-b712-196b835e971e admin admin] PATH_INFO: `/v3/users` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:05.924805 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89094351-d5c4-4ec0-b712-196b835e971e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:05.924805 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89094351-d5c4-4ec0-b712-196b835e971e admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:06.043893 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c718ba4b-923f-440c-89ed-05f32a92c594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:06.044743 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c718ba4b-923f-440c-89ed-05f32a92c594 admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:06.045341 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c718ba4b-923f-440c-89ed-05f32a92c594 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:06.045727 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c718ba4b-923f-440c-89ed-05f32a92c594 admin admin] PATH_INFO: `/v3/users` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:06.046665 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c718ba4b-923f-440c-89ed-05f32a92c594 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:06.049566 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c718ba4b-923f-440c-89ed-05f32a92c594 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:06.071840 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 891/1793] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 08:56:05 2022] POST /identity/v3/users => generated 300 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:56:06.074859 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89094351-d5c4-4ec0-b712-196b835e971e admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:06.112769 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89094351-d5c4-4ec0-b712-196b835e971e admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:06.129874 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45cf58d-fd0b-4a48-8c1e-7a95437d9325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:06.131599 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45cf58d-fd0b-4a48-8c1e-7a95437d9325 admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:06.132351 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45cf58d-fd0b-4a48-8c1e-7a95437d9325 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:06.132910 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45cf58d-fd0b-4a48-8c1e-7a95437d9325 admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:06.134109 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45cf58d-fd0b-4a48-8c1e-7a95437d9325 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:06.137277 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45cf58d-fd0b-4a48-8c1e-7a95437d9325 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:06.163134 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 892/1794] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:56:06 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:56:06.164833 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45cf58d-fd0b-4a48-8c1e-7a95437d9325 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:06.201919 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45cf58d-fd0b-4a48-8c1e-7a95437d9325 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:06.202761 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 903/1795] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 08:56:05 2022] POST /identity/v3/users => generated 305 bytes in 515 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:56:06.212100 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a224860-7cdf-46a1-8a88-d1af7e1c6601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:06.212100 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a224860-7cdf-46a1-8a88-d1af7e1c6601 admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:06.212100 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a224860-7cdf-46a1-8a88-d1af7e1c6601 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:06.212100 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a224860-7cdf-46a1-8a88-d1af7e1c6601 admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:06.212425 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a224860-7cdf-46a1-8a88-d1af7e1c6601 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:06.214418 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a224860-7cdf-46a1-8a88-d1af7e1c6601 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:06.217074 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c718ba4b-923f-440c-89ed-05f32a92c594 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:06.228593 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 893/1796] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:56:06 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:56:06.235658 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a224860-7cdf-46a1-8a88-d1af7e1c6601 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:06.260755 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a224860-7cdf-46a1-8a88-d1af7e1c6601 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:06.267869 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c718ba4b-923f-440c-89ed-05f32a92c594 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:06.270980 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5de6e4-fdfe-44d2-b992-653501d89235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:06.286411 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5de6e4-fdfe-44d2-b992-653501d89235 admin admin] REQUEST_METHOD: `PUT` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:06.286411 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5de6e4-fdfe-44d2-b992-653501d89235 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:06.286411 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5de6e4-fdfe-44d2-b992-653501d89235 admin admin] PATH_INFO: `/v3/projects/550854d2f6c44ecea91019e7ced90ed1/users/cdaab116afcc4ba086b55d3af50729f7/roles/b674fd0a592e4e0cada811bba5ceea4d` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:06.286999 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd09b35a-a3de-480f-b057-2d923e7b246c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:06.286999 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd09b35a-a3de-480f-b057-2d923e7b246c admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:06.286999 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd09b35a-a3de-480f-b057-2d923e7b246c admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:06.286999 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd09b35a-a3de-480f-b057-2d923e7b246c admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:06.287311 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd09b35a-a3de-480f-b057-2d923e7b246c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:06.287311 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd09b35a-a3de-480f-b057-2d923e7b246c admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:06.323915 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 904/1797] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:56:06 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:56:06.334281 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5de6e4-fdfe-44d2-b992-653501d89235 admin admin] RBAC: Authorizing `identity:create_grant(project_id=550854d2f6c44ecea91019e7ced90ed1, user_id=cdaab116afcc4ba086b55d3af50729f7, role_id=b674fd0a592e4e0cada811bba5ceea4d)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:06.340574 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5de6e4-fdfe-44d2-b992-653501d89235 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:06.342663 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd09b35a-a3de-480f-b057-2d923e7b246c admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:06.374872 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd09b35a-a3de-480f-b057-2d923e7b246c admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:06.396335 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9255e9-4ddb-48b8-96a8-58773e55e867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:06.397605 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9255e9-4ddb-48b8-96a8-58773e55e867 admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:06.398179 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9255e9-4ddb-48b8-96a8-58773e55e867 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:06.398975 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9255e9-4ddb-48b8-96a8-58773e55e867 admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:06.400290 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9255e9-4ddb-48b8-96a8-58773e55e867 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:06.405609 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9255e9-4ddb-48b8-96a8-58773e55e867 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:06.449450 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 905/1798] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:56:06 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:56:06.450742 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9255e9-4ddb-48b8-96a8-58773e55e867 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:06.483329 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9255e9-4ddb-48b8-96a8-58773e55e867 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:06.513760 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfc465d-d231-4eb9-9241-4e7008af568f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:06.514504 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfc465d-d231-4eb9-9241-4e7008af568f admin admin] REQUEST_METHOD: `PUT` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:06.514760 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfc465d-d231-4eb9-9241-4e7008af568f admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:06.515038 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfc465d-d231-4eb9-9241-4e7008af568f admin admin] PATH_INFO: `/v3/projects/70965096927f42e08425bc6a4a57f09c/users/76de3203581b45a6b71dc146b0130003/roles/b674fd0a592e4e0cada811bba5ceea4d` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:06.535931 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfc465d-d231-4eb9-9241-4e7008af568f admin admin] RBAC: Authorizing `identity:create_grant(project_id=70965096927f42e08425bc6a4a57f09c, user_id=76de3203581b45a6b71dc146b0130003, role_id=b674fd0a592e4e0cada811bba5ceea4d)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:06.539327 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfc465d-d231-4eb9-9241-4e7008af568f admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:06.554291 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 894/1799] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 08:56:06 2022] PUT /identity/v3/projects/550854d2f6c44ecea91019e7ced90ed1/users/cdaab116afcc4ba086b55d3af50729f7/roles/b674fd0a592e4e0cada811bba5ceea4d => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:56:06.574884 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8282290-0029-40ef-a0d1-1cf11134f633 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:06.576761 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8282290-0029-40ef-a0d1-1cf11134f633 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:06.577339 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8282290-0029-40ef-a0d1-1cf11134f633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:06.626072 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 906/1800] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 08:56:06 2022] PUT /identity/v3/projects/70965096927f42e08425bc6a4a57f09c/users/76de3203581b45a6b71dc146b0130003/roles/b674fd0a592e4e0cada811bba5ceea4d => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:06.638719 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-c8282290-0029-40ef-a0d1-1cf11134f633 None None] MFA Rules not processed for user `cdaab116afcc4ba086b55d3af50729f7`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:06.641579 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-c8282290-0029-40ef-a0d1-1cf11134f633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:06.654287 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18979b2d-25fe-41f2-802a-b7cc5aa4302d None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:06.654965 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18979b2d-25fe-41f2-802a-b7cc5aa4302d None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:06.655716 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18979b2d-25fe-41f2-802a-b7cc5aa4302d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:06.712284 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-18979b2d-25fe-41f2-802a-b7cc5aa4302d None None] MFA Rules not processed for user `76de3203581b45a6b71dc146b0130003`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:06.714793 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-18979b2d-25fe-41f2-802a-b7cc5aa4302d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:06.802085 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:06.873783 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 895/1801] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:56:06 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:06.876226 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8282290-0029-40ef-a0d1-1cf11134f633 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:06.928693 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 907/1802] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:56:06 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:06.931162 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18979b2d-25fe-41f2-802a-b7cc5aa4302d None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:07.151562 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8282290-0029-40ef-a0d1-1cf11134f633 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:07.174858 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e673c2c4-2ced-4cfb-ae64-1ebd0c99241a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:07.176136 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e673c2c4-2ced-4cfb-ae64-1ebd0c99241a admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:07.177105 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e673c2c4-2ced-4cfb-ae64-1ebd0c99241a admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:07.178399 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e673c2c4-2ced-4cfb-ae64-1ebd0c99241a admin admin] PATH_INFO: `/v3/projects` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:07.179348 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e673c2c4-2ced-4cfb-ae64-1ebd0c99241a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:07.188477 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e673c2c4-2ced-4cfb-ae64-1ebd0c99241a admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:07.262959 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18979b2d-25fe-41f2-802a-b7cc5aa4302d None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:07.279464 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109f9374-1641-44db-af61-31444ce18f8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:07.283121 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109f9374-1641-44db-af61-31444ce18f8e admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:07.283539 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109f9374-1641-44db-af61-31444ce18f8e admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:07.285537 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109f9374-1641-44db-af61-31444ce18f8e admin admin] PATH_INFO: `/v3/projects` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:07.286409 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109f9374-1641-44db-af61-31444ce18f8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:07.299115 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109f9374-1641-44db-af61-31444ce18f8e admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:07.509021 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 896/1803] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 08:56:06 2022] POST /identity/v3/projects => generated 399 bytes in 633 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:56:07.514630 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e673c2c4-2ced-4cfb-ae64-1ebd0c99241a admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:07.570436 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 908/1804] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 08:56:06 2022] POST /identity/v3/projects => generated 407 bytes in 635 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:56:07.572266 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109f9374-1641-44db-af61-31444ce18f8e admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:07.612443 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:07.773803 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e673c2c4-2ced-4cfb-ae64-1ebd0c99241a admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:07.937641 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109f9374-1641-44db-af61-31444ce18f8e admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:07.956254 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd37d78-0cad-40df-bdb2-e60cb63cb0a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:07.957545 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd37d78-0cad-40df-bdb2-e60cb63cb0a7 admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:07.959868 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd37d78-0cad-40df-bdb2-e60cb63cb0a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:07.960640 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd37d78-0cad-40df-bdb2-e60cb63cb0a7 admin admin] PATH_INFO: `/v3/users` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:07.964269 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd37d78-0cad-40df-bdb2-e60cb63cb0a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:07.970018 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd37d78-0cad-40df-bdb2-e60cb63cb0a7 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:07.976308 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c468fe-00da-410d-aa36-2660974f2700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:07.979332 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c468fe-00da-410d-aa36-2660974f2700 admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:07.979988 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c468fe-00da-410d-aa36-2660974f2700 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:07.980421 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c468fe-00da-410d-aa36-2660974f2700 admin admin] PATH_INFO: `/v3/users` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:07.984921 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c468fe-00da-410d-aa36-2660974f2700 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:07.989117 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c468fe-00da-410d-aa36-2660974f2700 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:08.155407 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 909/1805] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 08:56:07 2022] POST /identity/v3/users => generated 310 bytes in 584 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:56:08.170284 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd37d78-0cad-40df-bdb2-e60cb63cb0a7 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:08.216484 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd37d78-0cad-40df-bdb2-e60cb63cb0a7 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:08.230749 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 897/1806] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 08:56:07 2022] POST /identity/v3/users => generated 306 bytes in 721 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:56:08.237879 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a2b911-9f0b-4665-a611-c99de6541f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:08.238871 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a2b911-9f0b-4665-a611-c99de6541f6e admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:08.241364 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a2b911-9f0b-4665-a611-c99de6541f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:08.241723 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a2b911-9f0b-4665-a611-c99de6541f6e admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:08.249238 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a2b911-9f0b-4665-a611-c99de6541f6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:08.249713 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c468fe-00da-410d-aa36-2660974f2700 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:08.277764 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a2b911-9f0b-4665-a611-c99de6541f6e admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:08.304578 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c468fe-00da-410d-aa36-2660974f2700 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:08.311810 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eab8c46-d2a0-429b-8481-9098da584fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:08.312638 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eab8c46-d2a0-429b-8481-9098da584fec admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:08.312976 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eab8c46-d2a0-429b-8481-9098da584fec admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:08.313268 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eab8c46-d2a0-429b-8481-9098da584fec admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:08.313992 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eab8c46-d2a0-429b-8481-9098da584fec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:08.315947 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eab8c46-d2a0-429b-8481-9098da584fec admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:08.334267 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 910/1807] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:56:08 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:56:08.340880 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a2b911-9f0b-4665-a611-c99de6541f6e admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:08.345203 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 898/1808] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:56:08 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:56:08.351835 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eab8c46-d2a0-429b-8481-9098da584fec admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:08.384150 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eab8c46-d2a0-429b-8481-9098da584fec admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:08.387915 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a2b911-9f0b-4665-a611-c99de6541f6e admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:08.400932 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063c64d9-8721-4440-a387-ada8b8db9f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:08.400932 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063c64d9-8721-4440-a387-ada8b8db9f18 admin admin] REQUEST_METHOD: `PUT` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:08.400932 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063c64d9-8721-4440-a387-ada8b8db9f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:08.400932 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063c64d9-8721-4440-a387-ada8b8db9f18 admin admin] PATH_INFO: `/v3/projects/6d963f76ef4e419c936e6fab37a9ff32/users/5925fcdf681747c787abcf41e1bd7925/roles/b870cd1acaa744e1ae1d58b9ac283821` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:08.415131 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459cae05-2622-49d8-8cd0-987c25a9127a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:08.416289 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459cae05-2622-49d8-8cd0-987c25a9127a admin admin] REQUEST_METHOD: `PUT` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:08.416770 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459cae05-2622-49d8-8cd0-987c25a9127a admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:08.417194 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459cae05-2622-49d8-8cd0-987c25a9127a admin admin] PATH_INFO: `/v3/projects/3c4d03c0e40e4662b70eb56e03d240eb/users/ff4c84a3d3d84456801abd889b0d9155/roles/b870cd1acaa744e1ae1d58b9ac283821` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:08.431675 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063c64d9-8721-4440-a387-ada8b8db9f18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d963f76ef4e419c936e6fab37a9ff32, user_id=5925fcdf681747c787abcf41e1bd7925, role_id=b870cd1acaa744e1ae1d58b9ac283821)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:08.455096 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063c64d9-8721-4440-a387-ada8b8db9f18 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:08.476413 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459cae05-2622-49d8-8cd0-987c25a9127a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c4d03c0e40e4662b70eb56e03d240eb, user_id=ff4c84a3d3d84456801abd889b0d9155, role_id=b870cd1acaa744e1ae1d58b9ac283821)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:08.479315 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459cae05-2622-49d8-8cd0-987c25a9127a admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:08.645837 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 911/1809] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 08:56:08 2022] PUT /identity/v3/projects/3c4d03c0e40e4662b70eb56e03d240eb/users/ff4c84a3d3d84456801abd889b0d9155/roles/b870cd1acaa744e1ae1d58b9ac283821 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:08.655223 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459cae05-2622-49d8-8cd0-987c25a9127a admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:08.658640 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 899/1810] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 08:56:08 2022] PUT /identity/v3/projects/6d963f76ef4e419c936e6fab37a9ff32/users/5925fcdf681747c787abcf41e1bd7925/roles/b870cd1acaa744e1ae1d58b9ac283821 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:08.671698 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063c64d9-8721-4440-a387-ada8b8db9f18 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:08.811183 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459cae05-2622-49d8-8cd0-987c25a9127a admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:08.894469 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063c64d9-8721-4440-a387-ada8b8db9f18 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:08.917808 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba46596-805e-42b2-a441-f8fc3677ec43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:08.919086 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba46596-805e-42b2-a441-f8fc3677ec43 admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:08.919595 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba46596-805e-42b2-a441-f8fc3677ec43 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:08.920090 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba46596-805e-42b2-a441-f8fc3677ec43 admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:08.921347 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba46596-805e-42b2-a441-f8fc3677ec43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:08.924479 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba46596-805e-42b2-a441-f8fc3677ec43 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:08.960608 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 900/1811] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:56:08 2022] GET /identity/v3/roles => generated 1420 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:56:08.965883 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba46596-805e-42b2-a441-f8fc3677ec43 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:08.981927 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba46596-805e-42b2-a441-f8fc3677ec43 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:08.989153 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aadaca-f354-48e7-9bd1-0ae18b667531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:08.990678 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aadaca-f354-48e7-9bd1-0ae18b667531 admin admin] REQUEST_METHOD: `PUT` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:08.991135 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aadaca-f354-48e7-9bd1-0ae18b667531 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:08.991454 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aadaca-f354-48e7-9bd1-0ae18b667531 admin admin] PATH_INFO: `/v3/projects/6d963f76ef4e419c936e6fab37a9ff32/users/5925fcdf681747c787abcf41e1bd7925/roles/b870cd1acaa744e1ae1d58b9ac283821` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:08.995097 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99aadaca-f354-48e7-9bd1-0ae18b667531 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d963f76ef4e419c936e6fab37a9ff32, user_id=5925fcdf681747c787abcf41e1bd7925, role_id=b870cd1acaa744e1ae1d58b9ac283821)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:09.000412 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99aadaca-f354-48e7-9bd1-0ae18b667531 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:09.009620 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:09.065184 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 901/1812] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 08:56:08 2022] PUT /identity/v3/projects/6d963f76ef4e419c936e6fab37a9ff32/users/5925fcdf681747c787abcf41e1bd7925/roles/b870cd1acaa744e1ae1d58b9ac283821 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:56:09.083796 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268fc9a4-2fee-415e-9387-f84206fb5726 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:09.084067 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268fc9a4-2fee-415e-9387-f84206fb5726 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:09.084402 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268fc9a4-2fee-415e-9387-f84206fb5726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:09.087192 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02815eca-672e-4b50-854f-0b42b5988c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:09.087742 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02815eca-672e-4b50-854f-0b42b5988c4b admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:09.088027 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02815eca-672e-4b50-854f-0b42b5988c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:09.088306 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02815eca-672e-4b50-854f-0b42b5988c4b admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:09.089023 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02815eca-672e-4b50-854f-0b42b5988c4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:09.100572 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02815eca-672e-4b50-854f-0b42b5988c4b admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:09.129255 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 912/1813] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:56:08 2022] GET /identity/v3/roles => generated 1420 bytes in 477 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:56:09.133905 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02815eca-672e-4b50-854f-0b42b5988c4b admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:09.149929 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-268fc9a4-2fee-415e-9387-f84206fb5726 None None] MFA Rules not processed for user `5925fcdf681747c787abcf41e1bd7925`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:09.151502 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-268fc9a4-2fee-415e-9387-f84206fb5726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:09.170548 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02815eca-672e-4b50-854f-0b42b5988c4b admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:09.309986 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 902/1814] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:56:09 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:09.338310 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268fc9a4-2fee-415e-9387-f84206fb5726 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:09.346022 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-268fc9a4-2fee-415e-9387-f84206fb5726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:09.448629 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:09.453618 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4531e1d5-98d2-43cc-a449-c6e31073d8af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:09.463947 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4531e1d5-98d2-43cc-a449-c6e31073d8af admin admin] REQUEST_METHOD: `PUT` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:09.465796 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4531e1d5-98d2-43cc-a449-c6e31073d8af admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:09.465796 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4531e1d5-98d2-43cc-a449-c6e31073d8af admin admin] PATH_INFO: `/v3/projects/3c4d03c0e40e4662b70eb56e03d240eb/users/ff4c84a3d3d84456801abd889b0d9155/roles/b870cd1acaa744e1ae1d58b9ac283821` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:09.483323 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4531e1d5-98d2-43cc-a449-c6e31073d8af admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c4d03c0e40e4662b70eb56e03d240eb, user_id=ff4c84a3d3d84456801abd889b0d9155, role_id=b870cd1acaa744e1ae1d58b9ac283821)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:09.503250 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4531e1d5-98d2-43cc-a449-c6e31073d8af admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:09.550809 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 913/1815] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 08:56:09 2022] PUT /identity/v3/projects/3c4d03c0e40e4662b70eb56e03d240eb/users/ff4c84a3d3d84456801abd889b0d9155/roles/b870cd1acaa744e1ae1d58b9ac283821 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:09.574781 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af65669-7553-447d-b220-29fded7f7c05 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:09.575412 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af65669-7553-447d-b220-29fded7f7c05 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:09.575988 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af65669-7553-447d-b220-29fded7f7c05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:09.680604 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-5af65669-7553-447d-b220-29fded7f7c05 None None] MFA Rules not processed for user `ff4c84a3d3d84456801abd889b0d9155`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:09.683465 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-5af65669-7553-447d-b220-29fded7f7c05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:09.711218 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268fc9a4-2fee-415e-9387-f84206fb5726 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:09.919590 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a949f8d-2040-44ca-8033-b62d69cd408b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbb9ac667f04c35a63b2689df18efed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:09.923256 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a949f8d-2040-44ca-8033-b62d69cd408b service cinder] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:09.923910 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a949f8d-2040-44ca-8033-b62d69cd408b service cinder] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:09.928189 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a949f8d-2040-44ca-8033-b62d69cd408b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:09.969526 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 914/1816] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:56:09 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 391 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:10.170006 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a949f8d-2040-44ca-8033-b62d69cd408b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:10.175882 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a949f8d-2040-44ca-8033-b62d69cd408b service cinder] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:10.238895 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 903/1817] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:56:09 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 901 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:56:10.495240 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af65669-7553-447d-b220-29fded7f7c05 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:10.520170 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af65669-7553-447d-b220-29fded7f7c05 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:10.533755 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5eb694-3dbf-4b67-9250-f7eb17ebb578 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbb9ac667f04c35a63b2689df18efed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:10.534878 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5eb694-3dbf-4b67-9250-f7eb17ebb578 service cinder] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:10.535399 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5eb694-3dbf-4b67-9250-f7eb17ebb578 service cinder] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:10.535935 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5eb694-3dbf-4b67-9250-f7eb17ebb578 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:10.551857 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5eb694-3dbf-4b67-9250-f7eb17ebb578 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:10.557129 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5eb694-3dbf-4b67-9250-f7eb17ebb578 service cinder] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:10.584856 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 915/1818] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:56:10 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:56:10.942819 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a949f8d-2040-44ca-8033-b62d69cd408b service cinder] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:10.965112 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a949f8d-2040-44ca-8033-b62d69cd408b service cinder] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:10.975352 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99ca90f-056c-4f22-a0a2-653169f47be6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbb9ac667f04c35a63b2689df18efed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:10.976178 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99ca90f-056c-4f22-a0a2-653169f47be6 service cinder] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:10.979335 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99ca90f-056c-4f22-a0a2-653169f47be6 service cinder] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:10.979828 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99ca90f-056c-4f22-a0a2-653169f47be6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:10.988110 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:11.165773 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:11.237214 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99ca90f-056c-4f22-a0a2-653169f47be6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:11.239138 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99ca90f-056c-4f22-a0a2-653169f47be6 service cinder] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:11.277972 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e46498-592e-472d-8822-0aaf0b0b70a2 None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:11.280431 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e46498-592e-472d-8822-0aaf0b0b70a2 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:11.281304 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e46498-592e-472d-8822-0aaf0b0b70a2 None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:11.287588 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 904/1819] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:56:10 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:56:11.304852 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 916/1820] 10.222.0.55 () {54 vars in 921 bytes} [Wed Aug 24 08:56:11 2022] GET /identity => generated 270 bytes in 31 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:56:11.399835 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597d99da-fe9e-4bc7-bacf-300bec718bbd None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:11.400135 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597d99da-fe9e-4bc7-bacf-300bec718bbd None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:11.400393 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597d99da-fe9e-4bc7-bacf-300bec718bbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:11.452102 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-597d99da-fe9e-4bc7-bacf-300bec718bbd None None] MFA Rules not processed for user `f17da357e555420799ee681bffbcc268`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:11.453533 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-597d99da-fe9e-4bc7-bacf-300bec718bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:11.612265 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e46498-592e-472d-8822-0aaf0b0b70a2 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:11.618184 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-17e46498-592e-472d-8822-0aaf0b0b70a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:11.655726 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 905/1821] 10.222.0.55 () {58 vars in 1018 bytes} [Wed Aug 24 08:56:11 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:11.668749 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e46498-592e-472d-8822-0aaf0b0b70a2 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:11.673347 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bbd4fe-5fd4-45c3-8ba7-173099d28a3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbb9ac667f04c35a63b2689df18efed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:11.674146 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bbd4fe-5fd4-45c3-8ba7-173099d28a3b service cinder] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:11.674491 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bbd4fe-5fd4-45c3-8ba7-173099d28a3b service cinder] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:11.674835 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bbd4fe-5fd4-45c3-8ba7-173099d28a3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:11.691183 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597d99da-fe9e-4bc7-bacf-300bec718bbd None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:11.694759 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-597d99da-fe9e-4bc7-bacf-300bec718bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:11.719761 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597d99da-fe9e-4bc7-bacf-300bec718bbd None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:11.727446 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2451f6ee-b5d0-4a41-a63c-d157f5887c79 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17da357e555420799ee681bffbcc268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:11.729381 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2451f6ee-b5d0-4a41-a63c-d157f5887c79 service nova] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:11.729862 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2451f6ee-b5d0-4a41-a63c-d157f5887c79 service nova] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:11.730349 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2451f6ee-b5d0-4a41-a63c-d157f5887c79 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:11.744003 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2451f6ee-b5d0-4a41-a63c-d157f5887c79 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:11.746968 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2451f6ee-b5d0-4a41-a63c-d157f5887c79 service nova] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:11.761744 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 906/1822] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:56:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:56:11.820787 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bbd4fe-5fd4-45c3-8ba7-173099d28a3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:11.823426 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bbd4fe-5fd4-45c3-8ba7-173099d28a3b service cinder] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:11.837429 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 917/1823] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:56:11 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:56:12.383529 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:22.851856 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=98 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:23.737498 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=41 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:31.075629 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2451f6ee-b5d0-4a41-a63c-d157f5887c79 service nova] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:31.307839 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2451f6ee-b5d0-4a41-a63c-d157f5887c79 service nova] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:31.313684 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b4f666-a10c-40cc-ba0a-c649e3ab184c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:31.314445 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b4f666-a10c-40cc-ba0a-c649e3ab184c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:31.315064 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b4f666-a10c-40cc-ba0a-c649e3ab184c admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:31.315399 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b4f666-a10c-40cc-ba0a-c649e3ab184c admin admin] PATH_INFO: `/v3/users/cdaab116afcc4ba086b55d3af50729f7` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:31.317042 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b4f666-a10c-40cc-ba0a-c649e3ab184c admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdaab116afcc4ba086b55d3af50729f7)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:31.319494 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b4f666-a10c-40cc-ba0a-c649e3ab184c admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:31.645990 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-c7b4f666-a10c-40cc-ba0a-c649e3ab184c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdaab116afcc4ba086b55d3af50729f7'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:31.648310 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-c7b4f666-a10c-40cc-ba0a-c649e3ab184c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdaab116afcc4ba086b55d3af50729f7'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:31.649543 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-c7b4f666-a10c-40cc-ba0a-c649e3ab184c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdaab116afcc4ba086b55d3af50729f7'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:31.692840 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-c7b4f666-a10c-40cc-ba0a-c649e3ab184c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdaab116afcc4ba086b55d3af50729f7'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:31.831957 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-c7b4f666-a10c-40cc-ba0a-c649e3ab184c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdaab116afcc4ba086b55d3af50729f7'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:31.913989 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 907/1824] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 08:56:31 2022] DELETE /identity/v3/users/cdaab116afcc4ba086b55d3af50729f7 => generated 0 bytes in 840 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:31.922677 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919eecad-e330-4da1-b67d-2596a484542c None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:31.923665 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919eecad-e330-4da1-b67d-2596a484542c None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:31.924450 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919eecad-e330-4da1-b67d-2596a484542c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:32.014584 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-919eecad-e330-4da1-b67d-2596a484542c None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:32.016184 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-919eecad-e330-4da1-b67d-2596a484542c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:32.113939 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 918/1825] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:56:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:56:32.189412 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b4f666-a10c-40cc-ba0a-c649e3ab184c admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:32.229359 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-c7b4f666-a10c-40cc-ba0a-c649e3ab184c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:32.322741 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b4f666-a10c-40cc-ba0a-c649e3ab184c admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:32.328399 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d404310b-72b1-484e-832d-34f7ad559393 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:32.329697 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d404310b-72b1-484e-832d-34f7ad559393 service neutron] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:32.330262 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d404310b-72b1-484e-832d-34f7ad559393 service neutron] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:32.330748 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d404310b-72b1-484e-832d-34f7ad559393 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:32.343832 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d404310b-72b1-484e-832d-34f7ad559393 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:32.346364 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d404310b-72b1-484e-832d-34f7ad559393 service neutron] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:32.361157 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 908/1826] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:56:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:56:33.516966 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919eecad-e330-4da1-b67d-2596a484542c None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:33.519758 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-919eecad-e330-4da1-b67d-2596a484542c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:33.536132 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919eecad-e330-4da1-b67d-2596a484542c None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:33.542026 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c4edd0-d20a-405a-9a16-72a562d3443b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:33.542932 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c4edd0-d20a-405a-9a16-72a562d3443b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:33.543295 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c4edd0-d20a-405a-9a16-72a562d3443b admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:33.543680 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c4edd0-d20a-405a-9a16-72a562d3443b admin admin] PATH_INFO: `/v3/users/76de3203581b45a6b71dc146b0130003` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:33.545357 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c4edd0-d20a-405a-9a16-72a562d3443b admin admin] RBAC: Authorizing `identity:delete_user(user_id=76de3203581b45a6b71dc146b0130003)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:33.547958 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c4edd0-d20a-405a-9a16-72a562d3443b admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:33.849242 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-14c4edd0-d20a-405a-9a16-72a562d3443b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76de3203581b45a6b71dc146b0130003'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:33.851723 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-14c4edd0-d20a-405a-9a16-72a562d3443b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76de3203581b45a6b71dc146b0130003'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:33.853127 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-14c4edd0-d20a-405a-9a16-72a562d3443b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76de3203581b45a6b71dc146b0130003'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:33.905183 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-14c4edd0-d20a-405a-9a16-72a562d3443b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76de3203581b45a6b71dc146b0130003'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:34.008628 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-14c4edd0-d20a-405a-9a16-72a562d3443b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76de3203581b45a6b71dc146b0130003'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:34.031117 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 919/1827] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 08:56:33 2022] DELETE /identity/v3/users/76de3203581b45a6b71dc146b0130003 => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:34.037644 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9ef2a4-5c4e-4a86-a006-a569867f9c20 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:34.038762 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9ef2a4-5c4e-4a86-a006-a569867f9c20 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:34.039303 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9ef2a4-5c4e-4a86-a006-a569867f9c20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:34.067810 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-ae9ef2a4-5c4e-4a86-a006-a569867f9c20 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:34.069919 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-ae9ef2a4-5c4e-4a86-a006-a569867f9c20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:34.109378 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:34.274369 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 909/1828] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:56:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:34.493357 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c4edd0-d20a-405a-9a16-72a562d3443b admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:34.498936 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-14c4edd0-d20a-405a-9a16-72a562d3443b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:34.660284 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c4edd0-d20a-405a-9a16-72a562d3443b admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:34.669554 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a89abb-c2ce-4b84-938f-b79a47c35347 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:34.671214 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a89abb-c2ce-4b84-938f-b79a47c35347 service neutron] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:34.672000 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a89abb-c2ce-4b84-938f-b79a47c35347 service neutron] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:34.672827 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a89abb-c2ce-4b84-938f-b79a47c35347 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:34.699529 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a89abb-c2ce-4b84-938f-b79a47c35347 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:34.703553 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a89abb-c2ce-4b84-938f-b79a47c35347 service neutron] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:34.729080 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 920/1829] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:56:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:56:34.956066 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9ef2a4-5c4e-4a86-a006-a569867f9c20 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:34.958697 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-ae9ef2a4-5c4e-4a86-a006-a569867f9c20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:34.985969 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9ef2a4-5c4e-4a86-a006-a569867f9c20 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:34.995700 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ae90f7-e82f-4a3b-9329-e1f107bc686b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:34.996944 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ae90f7-e82f-4a3b-9329-e1f107bc686b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:34.997367 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ae90f7-e82f-4a3b-9329-e1f107bc686b admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:34.998693 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ae90f7-e82f-4a3b-9329-e1f107bc686b admin admin] PATH_INFO: `/v3/projects/550854d2f6c44ecea91019e7ced90ed1` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:35.001081 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ae90f7-e82f-4a3b-9329-e1f107bc686b admin admin] RBAC: Authorizing `identity:delete_project(project_id=550854d2f6c44ecea91019e7ced90ed1)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:35.005033 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ae90f7-e82f-4a3b-9329-e1f107bc686b admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:35.187852 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-45ae90f7-e82f-4a3b-9329-e1f107bc686b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '550854d2f6c44ecea91019e7ced90ed1', 'request_id': 'req-45ae90f7-e82f-4a3b-9329-e1f107bc686b'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:35.331233 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-45ae90f7-e82f-4a3b-9329-e1f107bc686b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '550854d2f6c44ecea91019e7ced90ed1', 'request_id': 'req-45ae90f7-e82f-4a3b-9329-e1f107bc686b'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:35.357706 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-45ae90f7-e82f-4a3b-9329-e1f107bc686b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '550854d2f6c44ecea91019e7ced90ed1', 'request_id': 'req-45ae90f7-e82f-4a3b-9329-e1f107bc686b'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:35.396748 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-45ae90f7-e82f-4a3b-9329-e1f107bc686b admin admin] The token cache is being invalidate because project 550854d2f6c44ecea91019e7ced90ed1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:56:35.397459 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-45ae90f7-e82f-4a3b-9329-e1f107bc686b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:35.398604 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-45ae90f7-e82f-4a3b-9329-e1f107bc686b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:35.400007 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6167b8-a452-45f9-bb39-6feb4bf2a8b1 None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:35.400614 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6167b8-a452-45f9-bb39-6feb4bf2a8b1 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:35.401181 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 910/1830] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 08:56:34 2022] DELETE /identity/v3/projects/550854d2f6c44ecea91019e7ced90ed1 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:35.401436 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6167b8-a452-45f9-bb39-6feb4bf2a8b1 None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:35.404935 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 921/1831] 10.222.0.55 () {54 vars in 921 bytes} [Wed Aug 24 08:56:35 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:56:35.406917 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ae90f7-e82f-4a3b-9329-e1f107bc686b admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:35.411126 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-45ae90f7-e82f-4a3b-9329-e1f107bc686b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:35.430415 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652f0865-e77a-4b40-a09b-f8bccb0f07ef None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:35.430993 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652f0865-e77a-4b40-a09b-f8bccb0f07ef None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:35.431557 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652f0865-e77a-4b40-a09b-f8bccb0f07ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:35.497135 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-652f0865-e77a-4b40-a09b-f8bccb0f07ef None None] MFA Rules not processed for user `f17da357e555420799ee681bffbcc268`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:35.503735 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-652f0865-e77a-4b40-a09b-f8bccb0f07ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:35.527632 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:35.576725 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ae90f7-e82f-4a3b-9329-e1f107bc686b admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:35.589266 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4914a4d8-f730-4eae-bd8e-8122426f1e65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:35.590879 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4914a4d8-f730-4eae-bd8e-8122426f1e65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:35.591159 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4914a4d8-f730-4eae-bd8e-8122426f1e65 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:35.591437 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4914a4d8-f730-4eae-bd8e-8122426f1e65 admin admin] PATH_INFO: `/v3/users/5925fcdf681747c787abcf41e1bd7925` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:35.595553 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4914a4d8-f730-4eae-bd8e-8122426f1e65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5925fcdf681747c787abcf41e1bd7925)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:35.604183 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4914a4d8-f730-4eae-bd8e-8122426f1e65 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:35.688830 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 922/1832] 10.222.0.55 () {58 vars in 1018 bytes} [Wed Aug 24 08:56:35 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:56:35.918525 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-4914a4d8-f730-4eae-bd8e-8122426f1e65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5925fcdf681747c787abcf41e1bd7925'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:35.920003 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-4914a4d8-f730-4eae-bd8e-8122426f1e65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5925fcdf681747c787abcf41e1bd7925'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:35.922374 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-4914a4d8-f730-4eae-bd8e-8122426f1e65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5925fcdf681747c787abcf41e1bd7925'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:35.930544 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:35.967085 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-4914a4d8-f730-4eae-bd8e-8122426f1e65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5925fcdf681747c787abcf41e1bd7925'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:36.077319 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652f0865-e77a-4b40-a09b-f8bccb0f07ef None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:36.079235 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-652f0865-e77a-4b40-a09b-f8bccb0f07ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:36.096431 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652f0865-e77a-4b40-a09b-f8bccb0f07ef None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:36.102447 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-4914a4d8-f730-4eae-bd8e-8122426f1e65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5925fcdf681747c787abcf41e1bd7925'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:36.105099 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e3dd61-5616-4e13-a908-7af708da2d1b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17da357e555420799ee681bffbcc268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:36.106443 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e3dd61-5616-4e13-a908-7af708da2d1b service nova] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:36.106951 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e3dd61-5616-4e13-a908-7af708da2d1b service nova] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:36.107732 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e3dd61-5616-4e13-a908-7af708da2d1b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:36.110163 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-c9e3dd61-5616-4e13-a908-7af708da2d1b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:36.126807 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e3dd61-5616-4e13-a908-7af708da2d1b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:36.130144 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e3dd61-5616-4e13-a908-7af708da2d1b service nova] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:36.138324 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 911/1833] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 08:56:35 2022] DELETE /identity/v3/users/5925fcdf681747c787abcf41e1bd7925 => generated 0 bytes in 731 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:36.168613 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 923/1834] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:56:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:56:36.967044 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4914a4d8-f730-4eae-bd8e-8122426f1e65 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:36.967991 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-4914a4d8-f730-4eae-bd8e-8122426f1e65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:37.078484 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4914a4d8-f730-4eae-bd8e-8122426f1e65 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:37.087057 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6edccd-0e74-4ff9-9e7e-c20c0eebd0fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:37.088177 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6edccd-0e74-4ff9-9e7e-c20c0eebd0fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:37.088932 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6edccd-0e74-4ff9-9e7e-c20c0eebd0fe admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:37.089365 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6edccd-0e74-4ff9-9e7e-c20c0eebd0fe admin admin] PATH_INFO: `/v3/users/78df990d90fe43e0bc3a0fd1cd834715` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:37.091438 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6edccd-0e74-4ff9-9e7e-c20c0eebd0fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=78df990d90fe43e0bc3a0fd1cd834715)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:37.095744 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6edccd-0e74-4ff9-9e7e-c20c0eebd0fe admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:37.109720 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e3dd61-5616-4e13-a908-7af708da2d1b service nova] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:37.119956 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-c9e3dd61-5616-4e13-a908-7af708da2d1b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:37.185228 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e3dd61-5616-4e13-a908-7af708da2d1b service nova] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:37.197390 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb16daf-cdcf-42f9-985c-aae5e7b14f8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:37.198837 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb16daf-cdcf-42f9-985c-aae5e7b14f8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:37.199452 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb16daf-cdcf-42f9-985c-aae5e7b14f8d admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:37.200127 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb16daf-cdcf-42f9-985c-aae5e7b14f8d admin admin] PATH_INFO: `/v3/projects/70965096927f42e08425bc6a4a57f09c` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:37.202996 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb16daf-cdcf-42f9-985c-aae5e7b14f8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=70965096927f42e08425bc6a4a57f09c)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:37.208322 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb16daf-cdcf-42f9-985c-aae5e7b14f8d admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:37.461690 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-cd6edccd-0e74-4ff9-9e7e-c20c0eebd0fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78df990d90fe43e0bc3a0fd1cd834715'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:37.465600 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-cd6edccd-0e74-4ff9-9e7e-c20c0eebd0fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78df990d90fe43e0bc3a0fd1cd834715'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:37.465849 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-5fb16daf-cdcf-42f9-985c-aae5e7b14f8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70965096927f42e08425bc6a4a57f09c', 'request_id': 'req-5fb16daf-cdcf-42f9-985c-aae5e7b14f8d'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:37.469336 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-cd6edccd-0e74-4ff9-9e7e-c20c0eebd0fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78df990d90fe43e0bc3a0fd1cd834715'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:37.478485 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-5fb16daf-cdcf-42f9-985c-aae5e7b14f8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70965096927f42e08425bc6a4a57f09c', 'request_id': 'req-5fb16daf-cdcf-42f9-985c-aae5e7b14f8d'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:37.511402 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-cd6edccd-0e74-4ff9-9e7e-c20c0eebd0fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78df990d90fe43e0bc3a0fd1cd834715'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:37.554826 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-5fb16daf-cdcf-42f9-985c-aae5e7b14f8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70965096927f42e08425bc6a4a57f09c', 'request_id': 'req-5fb16daf-cdcf-42f9-985c-aae5e7b14f8d'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:37.603757 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-5fb16daf-cdcf-42f9-985c-aae5e7b14f8d admin admin] The token cache is being invalidate because project 70965096927f42e08425bc6a4a57f09c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:56:37.604955 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-5fb16daf-cdcf-42f9-985c-aae5e7b14f8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:37.605391 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-5fb16daf-cdcf-42f9-985c-aae5e7b14f8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:37.607601 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 924/1835] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 08:56:37 2022] DELETE /identity/v3/projects/70965096927f42e08425bc6a4a57f09c => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:37.608805 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb16daf-cdcf-42f9-985c-aae5e7b14f8d admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:37.613162 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-5fb16daf-cdcf-42f9-985c-aae5e7b14f8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:37.627030 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-cd6edccd-0e74-4ff9-9e7e-c20c0eebd0fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78df990d90fe43e0bc3a0fd1cd834715'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:37.705005 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 912/1836] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 08:56:36 2022] DELETE /identity/v3/users/78df990d90fe43e0bc3a0fd1cd834715 => generated 0 bytes in 741 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:37.708232 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6edccd-0e74-4ff9-9e7e-c20c0eebd0fe admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:37.711799 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-cd6edccd-0e74-4ff9-9e7e-c20c0eebd0fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:37.758735 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb16daf-cdcf-42f9-985c-aae5e7b14f8d admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:37.822870 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:37.853723 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:37.882410 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6edccd-0e74-4ff9-9e7e-c20c0eebd0fe admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:37.895680 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d7c91b-d2fa-49bb-961d-7e5927da869e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:37.896782 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d7c91b-d2fa-49bb-961d-7e5927da869e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:37.897241 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d7c91b-d2fa-49bb-961d-7e5927da869e admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:37.898007 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d7c91b-d2fa-49bb-961d-7e5927da869e admin admin] PATH_INFO: `/v3/users/ff4c84a3d3d84456801abd889b0d9155` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:37.900270 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d7c91b-d2fa-49bb-961d-7e5927da869e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff4c84a3d3d84456801abd889b0d9155)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:37.904737 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d7c91b-d2fa-49bb-961d-7e5927da869e admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:37.952357 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce97b7bf-4ab0-4801-a1a4-0bbef3650af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:37.952988 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce97b7bf-4ab0-4801-a1a4-0bbef3650af6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:37.953456 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce97b7bf-4ab0-4801-a1a4-0bbef3650af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:37.953937 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce97b7bf-4ab0-4801-a1a4-0bbef3650af6 admin admin] PATH_INFO: `/v3/projects/6d963f76ef4e419c936e6fab37a9ff32` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:37.955664 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce97b7bf-4ab0-4801-a1a4-0bbef3650af6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d963f76ef4e419c936e6fab37a9ff32)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:37.958614 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce97b7bf-4ab0-4801-a1a4-0bbef3650af6 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:38.144597 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-70d7c91b-d2fa-49bb-961d-7e5927da869e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff4c84a3d3d84456801abd889b0d9155'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:38.145936 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-70d7c91b-d2fa-49bb-961d-7e5927da869e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff4c84a3d3d84456801abd889b0d9155'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:38.146705 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-70d7c91b-d2fa-49bb-961d-7e5927da869e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff4c84a3d3d84456801abd889b0d9155'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:38.154635 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-ce97b7bf-4ab0-4801-a1a4-0bbef3650af6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d963f76ef4e419c936e6fab37a9ff32', 'request_id': 'req-ce97b7bf-4ab0-4801-a1a4-0bbef3650af6'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:38.162136 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-ce97b7bf-4ab0-4801-a1a4-0bbef3650af6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d963f76ef4e419c936e6fab37a9ff32', 'request_id': 'req-ce97b7bf-4ab0-4801-a1a4-0bbef3650af6'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:38.167003 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-70d7c91b-d2fa-49bb-961d-7e5927da869e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff4c84a3d3d84456801abd889b0d9155'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:38.272064 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-ce97b7bf-4ab0-4801-a1a4-0bbef3650af6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d963f76ef4e419c936e6fab37a9ff32', 'request_id': 'req-ce97b7bf-4ab0-4801-a1a4-0bbef3650af6'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:38.302652 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-ce97b7bf-4ab0-4801-a1a4-0bbef3650af6 admin admin] The token cache is being invalidate because project 6d963f76ef4e419c936e6fab37a9ff32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:56:38.304768 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-ce97b7bf-4ab0-4801-a1a4-0bbef3650af6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:38.307594 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-ce97b7bf-4ab0-4801-a1a4-0bbef3650af6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:38.319303 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 925/1837] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 08:56:37 2022] DELETE /identity/v3/projects/6d963f76ef4e419c936e6fab37a9ff32 => generated 0 bytes in 703 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:38.322118 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c93cf2b-d638-4a78-96fd-f3eac825e998 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:38.322610 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c93cf2b-d638-4a78-96fd-f3eac825e998 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:38.323090 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c93cf2b-d638-4a78-96fd-f3eac825e998 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:38.339590 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-70d7c91b-d2fa-49bb-961d-7e5927da869e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff4c84a3d3d84456801abd889b0d9155'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:38.359206 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-4c93cf2b-d638-4a78-96fd-f3eac825e998 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:38.362108 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-4c93cf2b-d638-4a78-96fd-f3eac825e998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:38.367907 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 913/1838] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 08:56:37 2022] DELETE /identity/v3/users/ff4c84a3d3d84456801abd889b0d9155 => generated 0 bytes in 662 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:38.370468 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71be72c5-031c-40d9-80c9-1fd2278a8137 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:38.370757 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71be72c5-031c-40d9-80c9-1fd2278a8137 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:38.371300 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71be72c5-031c-40d9-80c9-1fd2278a8137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:38.464188 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-71be72c5-031c-40d9-80c9-1fd2278a8137 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:38.470670 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-71be72c5-031c-40d9-80c9-1fd2278a8137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:38.542046 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 926/1839] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:56:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:38.565589 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c93cf2b-d638-4a78-96fd-f3eac825e998 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:38.570393 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-4c93cf2b-d638-4a78-96fd-f3eac825e998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:38.610184 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 914/1840] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:56:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:38.622699 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275e1bf9-761a-4627-8021-8cc602859a35 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:38.623240 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275e1bf9-761a-4627-8021-8cc602859a35 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:38.623566 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275e1bf9-761a-4627-8021-8cc602859a35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:38.687448 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-275e1bf9-761a-4627-8021-8cc602859a35 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:38.692264 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-275e1bf9-761a-4627-8021-8cc602859a35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:38.709627 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c93cf2b-d638-4a78-96fd-f3eac825e998 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:38.717915 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 915/1841] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:56:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:56:38.724658 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275e1bf9-761a-4627-8021-8cc602859a35 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:38.735834 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05e2d29-8748-4712-8404-c08556318f13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:38.738781 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e2d29-8748-4712-8404-c08556318f13 service neutron] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:38.739148 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e2d29-8748-4712-8404-c08556318f13 service neutron] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:38.741660 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275e1bf9-761a-4627-8021-8cc602859a35 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:38.741879 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e2d29-8748-4712-8404-c08556318f13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:38.756659 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b108b5f-b4ae-4fc5-abf3-39b16804d13d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:38.757366 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b108b5f-b4ae-4fc5-abf3-39b16804d13d admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:38.757641 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05e2d29-8748-4712-8404-c08556318f13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:38.757823 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b108b5f-b4ae-4fc5-abf3-39b16804d13d admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:38.757999 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b108b5f-b4ae-4fc5-abf3-39b16804d13d admin admin] PATH_INFO: `/v3/domains` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:38.758689 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b108b5f-b4ae-4fc5-abf3-39b16804d13d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:56:38.759130 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b108b5f-b4ae-4fc5-abf3-39b16804d13d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:38.759977 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05e2d29-8748-4712-8404-c08556318f13 service neutron] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:38.760835 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b108b5f-b4ae-4fc5-abf3-39b16804d13d admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:38.787081 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 916/1842] 10.222.0.55 () {58 vars in 1153 bytes} [Wed Aug 24 08:56:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:56:38.787373 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 927/1843] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:56:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:56:38.788394 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b108b5f-b4ae-4fc5-abf3-39b16804d13d admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:38.820171 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b108b5f-b4ae-4fc5-abf3-39b16804d13d admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:38.834749 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2e7c13-0697-4045-8795-53e641c9ff04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:38.835811 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2e7c13-0697-4045-8795-53e641c9ff04 admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:38.836292 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2e7c13-0697-4045-8795-53e641c9ff04 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:38.836852 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2e7c13-0697-4045-8795-53e641c9ff04 admin admin] PATH_INFO: `/v3/projects` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:38.838047 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2e7c13-0697-4045-8795-53e641c9ff04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:38.843202 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2e7c13-0697-4045-8795-53e641c9ff04 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:39.063359 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 917/1844] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 08:56:38 2022] POST /identity/v3/projects => generated 365 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:56:39.065442 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05e2d29-8748-4712-8404-c08556318f13 service neutron] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:39.092807 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05e2d29-8748-4712-8404-c08556318f13 service neutron] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:39.246680 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9983a40c-bd5d-42b2-a94d-b15072fbba00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:39.250190 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9983a40c-bd5d-42b2-a94d-b15072fbba00 admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:39.250706 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9983a40c-bd5d-42b2-a94d-b15072fbba00 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:39.251255 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9983a40c-bd5d-42b2-a94d-b15072fbba00 admin admin] PATH_INFO: `/v3/users` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:39.252752 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9983a40c-bd5d-42b2-a94d-b15072fbba00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:39.256991 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9983a40c-bd5d-42b2-a94d-b15072fbba00 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:39.393434 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:39.419790 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 928/1845] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 08:56:39 2022] POST /identity/v3/users => generated 283 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:56:39.439234 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2e7c13-0697-4045-8795-53e641c9ff04 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:39.484036 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2e7c13-0697-4045-8795-53e641c9ff04 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:39.484662 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c29f5fd-d86f-44b2-9aac-d62437069d88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:39.485916 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c29f5fd-d86f-44b2-9aac-d62437069d88 admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:39.486550 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c29f5fd-d86f-44b2-9aac-d62437069d88 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:39.487164 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c29f5fd-d86f-44b2-9aac-d62437069d88 admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:39.491093 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c29f5fd-d86f-44b2-9aac-d62437069d88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:39.495431 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c29f5fd-d86f-44b2-9aac-d62437069d88 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:39.529677 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 918/1846] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:56:39 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:56:39.531971 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9983a40c-bd5d-42b2-a94d-b15072fbba00 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:39.551814 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9983a40c-bd5d-42b2-a94d-b15072fbba00 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:39.561169 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11705a50-7466-4892-8599-f5cc0777d5c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:39.562561 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11705a50-7466-4892-8599-f5cc0777d5c0 admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:39.563413 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11705a50-7466-4892-8599-f5cc0777d5c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:39.564307 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11705a50-7466-4892-8599-f5cc0777d5c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:39.565949 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11705a50-7466-4892-8599-f5cc0777d5c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:39.569619 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11705a50-7466-4892-8599-f5cc0777d5c0 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:39.597087 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 929/1847] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:56:39 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:56:39.598493 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c29f5fd-d86f-44b2-9aac-d62437069d88 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:39.619151 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c29f5fd-d86f-44b2-9aac-d62437069d88 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:39.625936 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e2662c-d69f-4332-a303-31fb71637525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:39.627362 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e2662c-d69f-4332-a303-31fb71637525 admin admin] REQUEST_METHOD: `PUT` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:39.628115 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e2662c-d69f-4332-a303-31fb71637525 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:39.628784 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e2662c-d69f-4332-a303-31fb71637525 admin admin] PATH_INFO: `/v3/projects/b3c25dd5adae4846b4bbac08ad2d9a44/users/7c21c842aa3f4fcdab49136642a725c9/roles/b674fd0a592e4e0cada811bba5ceea4d` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:39.649388 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:39.668024 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e2662c-d69f-4332-a303-31fb71637525 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3c25dd5adae4846b4bbac08ad2d9a44, user_id=7c21c842aa3f4fcdab49136642a725c9, role_id=b674fd0a592e4e0cada811bba5ceea4d)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:39.681548 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e2662c-d69f-4332-a303-31fb71637525 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:39.823966 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 919/1848] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 08:56:39 2022] PUT /identity/v3/projects/b3c25dd5adae4846b4bbac08ad2d9a44/users/7c21c842aa3f4fcdab49136642a725c9/roles/b674fd0a592e4e0cada811bba5ceea4d => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:56:39.834956 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fe04c6-ffef-4288-a786-d4b5c75782a2 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:39.835271 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fe04c6-ffef-4288-a786-d4b5c75782a2 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:39.835510 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fe04c6-ffef-4288-a786-d4b5c75782a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:39.897843 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-d1fe04c6-ffef-4288-a786-d4b5c75782a2 None None] MFA Rules not processed for user `7c21c842aa3f4fcdab49136642a725c9`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:39.899943 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-d1fe04c6-ffef-4288-a786-d4b5c75782a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:40.045332 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 930/1849] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:56:39 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:40.050592 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e2662c-d69f-4332-a303-31fb71637525 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:40.066542 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fe04c6-ffef-4288-a786-d4b5c75782a2 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:40.070251 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-d1fe04c6-ffef-4288-a786-d4b5c75782a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:40.174196 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e2662c-d69f-4332-a303-31fb71637525 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:40.181670 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabb6c38-7908-4a3a-a64a-14c2c7569aa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:40.199284 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabb6c38-7908-4a3a-a64a-14c2c7569aa3 admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:40.200989 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabb6c38-7908-4a3a-a64a-14c2c7569aa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:40.201413 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabb6c38-7908-4a3a-a64a-14c2c7569aa3 admin admin] PATH_INFO: `/v3/projects` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:40.202347 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabb6c38-7908-4a3a-a64a-14c2c7569aa3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:40.209845 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabb6c38-7908-4a3a-a64a-14c2c7569aa3 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:40.273836 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fe04c6-ffef-4288-a786-d4b5c75782a2 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:40.287854 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a4d258-de16-4fea-a3bc-bc367cb150bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:40.288788 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a4d258-de16-4fea-a3bc-bc367cb150bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:40.291714 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a4d258-de16-4fea-a3bc-bc367cb150bb admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:40.295356 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a4d258-de16-4fea-a3bc-bc367cb150bb admin admin] PATH_INFO: `/v3/projects/d7650312b93a44ceb07ebd745fd83f05` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:40.295356 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a4d258-de16-4fea-a3bc-bc367cb150bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7650312b93a44ceb07ebd745fd83f05)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:40.299246 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a4d258-de16-4fea-a3bc-bc367cb150bb admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:40.331323 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:40.434962 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 920/1850] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 08:56:40 2022] POST /identity/v3/projects => generated 367 bytes in 385 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:56:40.443691 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabb6c38-7908-4a3a-a64a-14c2c7569aa3 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:40.541312 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-30a4d258-de16-4fea-a3bc-bc367cb150bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7650312b93a44ceb07ebd745fd83f05', 'request_id': 'req-30a4d258-de16-4fea-a3bc-bc367cb150bb'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:40.581067 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-30a4d258-de16-4fea-a3bc-bc367cb150bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7650312b93a44ceb07ebd745fd83f05', 'request_id': 'req-30a4d258-de16-4fea-a3bc-bc367cb150bb'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:40.623403 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabb6c38-7908-4a3a-a64a-14c2c7569aa3 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:40.688790 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-30a4d258-de16-4fea-a3bc-bc367cb150bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7650312b93a44ceb07ebd745fd83f05', 'request_id': 'req-30a4d258-de16-4fea-a3bc-bc367cb150bb'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:40.715849 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-30a4d258-de16-4fea-a3bc-bc367cb150bb admin admin] The token cache is being invalidate because project d7650312b93a44ceb07ebd745fd83f05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:56:40.716350 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-30a4d258-de16-4fea-a3bc-bc367cb150bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:40.717683 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-30a4d258-de16-4fea-a3bc-bc367cb150bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:40.724023 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 931/1851] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 08:56:40 2022] DELETE /identity/v3/projects/d7650312b93a44ceb07ebd745fd83f05 => generated 0 bytes in 656 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:40.726605 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a4d258-de16-4fea-a3bc-bc367cb150bb admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:40.729671 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-30a4d258-de16-4fea-a3bc-bc367cb150bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:40.753271 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1994bf-1dec-409f-9b45-6042e299e18f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:40.754820 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1994bf-1dec-409f-9b45-6042e299e18f admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:40.755895 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1994bf-1dec-409f-9b45-6042e299e18f admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:40.755895 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1994bf-1dec-409f-9b45-6042e299e18f admin admin] PATH_INFO: `/v3/users` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:40.757079 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1994bf-1dec-409f-9b45-6042e299e18f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:40.761361 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1994bf-1dec-409f-9b45-6042e299e18f admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:40.847294 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a4d258-de16-4fea-a3bc-bc367cb150bb admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:40.853214 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9c9618-33dc-47d3-a4e5-cb4f4f4aab79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:40.854009 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9c9618-33dc-47d3-a4e5-cb4f4f4aab79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:40.854401 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9c9618-33dc-47d3-a4e5-cb4f4f4aab79 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:40.854780 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9c9618-33dc-47d3-a4e5-cb4f4f4aab79 admin admin] PATH_INFO: `/v3/users/500945cbdd7f4812adf7c5ae48a42caf` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:40.856390 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9c9618-33dc-47d3-a4e5-cb4f4f4aab79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=500945cbdd7f4812adf7c5ae48a42caf)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:40.862368 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9c9618-33dc-47d3-a4e5-cb4f4f4aab79 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:41.012597 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 921/1852] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 08:56:40 2022] POST /identity/v3/users => generated 290 bytes in 569 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:56:41.029468 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1994bf-1dec-409f-9b45-6042e299e18f admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:41.032160 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-2e1994bf-1dec-409f-9b45-6042e299e18f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:41.068845 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1994bf-1dec-409f-9b45-6042e299e18f admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:41.075348 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cab30a-dd8f-465c-9ef5-5def0f06278f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:41.076401 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cab30a-dd8f-465c-9ef5-5def0f06278f admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:41.077390 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cab30a-dd8f-465c-9ef5-5def0f06278f admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:41.077873 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cab30a-dd8f-465c-9ef5-5def0f06278f admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:41.079914 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cab30a-dd8f-465c-9ef5-5def0f06278f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:41.083060 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cab30a-dd8f-465c-9ef5-5def0f06278f admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:41.093689 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:41.105273 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 922/1853] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:56:41 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:56:41.106664 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cab30a-dd8f-465c-9ef5-5def0f06278f admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:41.109908 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-c4cab30a-dd8f-465c-9ef5-5def0f06278f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:41.137854 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cab30a-dd8f-465c-9ef5-5def0f06278f admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:41.145552 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b27696-7484-451a-9e28-5dcc3f3b93c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:41.146749 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b27696-7484-451a-9e28-5dcc3f3b93c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:41.147239 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b27696-7484-451a-9e28-5dcc3f3b93c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:41.147695 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b27696-7484-451a-9e28-5dcc3f3b93c0 admin admin] PATH_INFO: `/v3/projects/3c4d03c0e40e4662b70eb56e03d240eb` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:41.149986 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b27696-7484-451a-9e28-5dcc3f3b93c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c4d03c0e40e4662b70eb56e03d240eb)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:41.154248 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b27696-7484-451a-9e28-5dcc3f3b93c0 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:41.275392 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-3b9c9618-33dc-47d3-a4e5-cb4f4f4aab79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '500945cbdd7f4812adf7c5ae48a42caf'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:41.278442 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-3b9c9618-33dc-47d3-a4e5-cb4f4f4aab79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '500945cbdd7f4812adf7c5ae48a42caf'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:41.279950 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-3b9c9618-33dc-47d3-a4e5-cb4f4f4aab79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '500945cbdd7f4812adf7c5ae48a42caf'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:41.311631 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-3b9c9618-33dc-47d3-a4e5-cb4f4f4aab79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '500945cbdd7f4812adf7c5ae48a42caf'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:41.402427 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-89b27696-7484-451a-9e28-5dcc3f3b93c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c4d03c0e40e4662b70eb56e03d240eb', 'request_id': 'req-89b27696-7484-451a-9e28-5dcc3f3b93c0'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:41.414145 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-3b9c9618-33dc-47d3-a4e5-cb4f4f4aab79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '500945cbdd7f4812adf7c5ae48a42caf'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:41.460164 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 932/1854] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 08:56:40 2022] DELETE /identity/v3/users/500945cbdd7f4812adf7c5ae48a42caf => generated 0 bytes in 735 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:41.461676 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9c9618-33dc-47d3-a4e5-cb4f4f4aab79 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:41.477075 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-3b9c9618-33dc-47d3-a4e5-cb4f4f4aab79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:41.511836 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-89b27696-7484-451a-9e28-5dcc3f3b93c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c4d03c0e40e4662b70eb56e03d240eb', 'request_id': 'req-89b27696-7484-451a-9e28-5dcc3f3b93c0'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:41.552491 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:41.553973 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-89b27696-7484-451a-9e28-5dcc3f3b93c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c4d03c0e40e4662b70eb56e03d240eb', 'request_id': 'req-89b27696-7484-451a-9e28-5dcc3f3b93c0'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:41.605373 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-89b27696-7484-451a-9e28-5dcc3f3b93c0 admin admin] The token cache is being invalidate because project 3c4d03c0e40e4662b70eb56e03d240eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:56:41.606050 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-89b27696-7484-451a-9e28-5dcc3f3b93c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:41.607492 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-89b27696-7484-451a-9e28-5dcc3f3b93c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:41.611794 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 923/1855] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 08:56:41 2022] DELETE /identity/v3/projects/3c4d03c0e40e4662b70eb56e03d240eb => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:41.622326 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638a1570-dd2a-45fd-bc9f-f37544328375 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:41.636418 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638a1570-dd2a-45fd-bc9f-f37544328375 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:41.636418 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638a1570-dd2a-45fd-bc9f-f37544328375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:41.664730 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-638a1570-dd2a-45fd-bc9f-f37544328375 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:41.667236 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-638a1570-dd2a-45fd-bc9f-f37544328375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:41.680060 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 924/1856] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:56:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:41.690408 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9c9618-33dc-47d3-a4e5-cb4f4f4aab79 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:41.692508 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8302b6ae-4084-4026-ab7e-67824051725d None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:41.692902 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8302b6ae-4084-4026-ab7e-67824051725d None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:41.693375 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8302b6ae-4084-4026-ab7e-67824051725d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:41.703183 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599cf1ac-695a-4b3d-957f-139212f8a424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:41.705335 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599cf1ac-695a-4b3d-957f-139212f8a424 admin admin] REQUEST_METHOD: `PUT` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:41.705702 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599cf1ac-695a-4b3d-957f-139212f8a424 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:41.706219 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599cf1ac-695a-4b3d-957f-139212f8a424 admin admin] PATH_INFO: `/v3/projects/204cfa7633604adabe6a2a3331405b53/users/f8238e09838d47d1bd59e0e7e7a0fab2/roles/b870cd1acaa744e1ae1d58b9ac283821` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:41.753128 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599cf1ac-695a-4b3d-957f-139212f8a424 admin admin] RBAC: Authorizing `identity:create_grant(project_id=204cfa7633604adabe6a2a3331405b53, user_id=f8238e09838d47d1bd59e0e7e7a0fab2, role_id=b870cd1acaa744e1ae1d58b9ac283821)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:41.759987 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599cf1ac-695a-4b3d-957f-139212f8a424 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:41.769326 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-8302b6ae-4084-4026-ab7e-67824051725d None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:41.771272 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-8302b6ae-4084-4026-ab7e-67824051725d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:41.792128 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 925/1857] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:56:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:56:41.797580 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8302b6ae-4084-4026-ab7e-67824051725d None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:41.821878 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8302b6ae-4084-4026-ab7e-67824051725d None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:41.831410 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb33bf1-689e-4c11-98f8-0f9faa1230f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:41.832071 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb33bf1-689e-4c11-98f8-0f9faa1230f2 admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:41.833029 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb33bf1-689e-4c11-98f8-0f9faa1230f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:41.833253 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb33bf1-689e-4c11-98f8-0f9faa1230f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:41.833941 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb33bf1-689e-4c11-98f8-0f9faa1230f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:56:41.834245 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb33bf1-689e-4c11-98f8-0f9faa1230f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:41.836010 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb33bf1-689e-4c11-98f8-0f9faa1230f2 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:41.870847 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 926/1858] 10.222.0.55 () {58 vars in 1153 bytes} [Wed Aug 24 08:56:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:56:41.871894 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb33bf1-689e-4c11-98f8-0f9faa1230f2 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:41.885042 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb33bf1-689e-4c11-98f8-0f9faa1230f2 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:41.905714 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 933/1859] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 08:56:41 2022] PUT /identity/v3/projects/204cfa7633604adabe6a2a3331405b53/users/f8238e09838d47d1bd59e0e7e7a0fab2/roles/b870cd1acaa744e1ae1d58b9ac283821 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:41.911459 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599cf1ac-695a-4b3d-957f-139212f8a424 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:41.916517 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-599cf1ac-695a-4b3d-957f-139212f8a424 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:42.051632 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba612d5-c4cf-4f65-a465-921ca94d304b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:42.052965 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba612d5-c4cf-4f65-a465-921ca94d304b admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:42.053435 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba612d5-c4cf-4f65-a465-921ca94d304b admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:42.054713 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba612d5-c4cf-4f65-a465-921ca94d304b admin admin] PATH_INFO: `/v3/projects` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:42.055778 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba612d5-c4cf-4f65-a465-921ca94d304b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:42.059405 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba612d5-c4cf-4f65-a465-921ca94d304b admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:42.141560 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599cf1ac-695a-4b3d-957f-139212f8a424 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:42.176069 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42360231-dbed-4b5c-b25a-778bb8ae93dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:42.177121 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42360231-dbed-4b5c-b25a-778bb8ae93dd admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:42.177653 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42360231-dbed-4b5c-b25a-778bb8ae93dd admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:42.179699 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42360231-dbed-4b5c-b25a-778bb8ae93dd admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:42.180808 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42360231-dbed-4b5c-b25a-778bb8ae93dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:42.184585 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42360231-dbed-4b5c-b25a-778bb8ae93dd admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:42.198659 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 934/1860] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:56:41 2022] GET /identity/v3/roles => generated 1420 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:56:42.205224 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42360231-dbed-4b5c-b25a-778bb8ae93dd admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:42.219688 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42360231-dbed-4b5c-b25a-778bb8ae93dd admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:42.227602 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bb7b88-c914-4960-8626-a2fab60179a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:42.229272 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bb7b88-c914-4960-8626-a2fab60179a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:42.230037 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bb7b88-c914-4960-8626-a2fab60179a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:42.230715 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bb7b88-c914-4960-8626-a2fab60179a0 admin admin] PATH_INFO: `/v3/projects/204cfa7633604adabe6a2a3331405b53/users/f8238e09838d47d1bd59e0e7e7a0fab2/roles/b870cd1acaa744e1ae1d58b9ac283821` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:42.233591 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bb7b88-c914-4960-8626-a2fab60179a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=204cfa7633604adabe6a2a3331405b53, user_id=f8238e09838d47d1bd59e0e7e7a0fab2, role_id=b870cd1acaa744e1ae1d58b9ac283821)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:42.237760 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bb7b88-c914-4960-8626-a2fab60179a0 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:42.262711 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 935/1861] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 08:56:42 2022] PUT /identity/v3/projects/204cfa7633604adabe6a2a3331405b53/users/f8238e09838d47d1bd59e0e7e7a0fab2/roles/b870cd1acaa744e1ae1d58b9ac283821 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:42.283478 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a51a85-f1c1-46fb-8c85-830340ee8f8c None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:42.284450 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a51a85-f1c1-46fb-8c85-830340ee8f8c None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:42.284450 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a51a85-f1c1-46fb-8c85-830340ee8f8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:42.298158 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 927/1862] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 08:56:41 2022] POST /identity/v3/projects => generated 361 bytes in 427 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:56:42.308131 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba612d5-c4cf-4f65-a465-921ca94d304b admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:42.336210 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-18a51a85-f1c1-46fb-8c85-830340ee8f8c None None] MFA Rules not processed for user `f8238e09838d47d1bd59e0e7e7a0fab2`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:42.338016 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-18a51a85-f1c1-46fb-8c85-830340ee8f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:42.390855 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:42.413873 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba612d5-c4cf-4f65-a465-921ca94d304b admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:42.418776 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636b920b-6506-444e-8edc-15bdf6d0b18e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:42.419430 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636b920b-6506-444e-8edc-15bdf6d0b18e admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:42.419714 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636b920b-6506-444e-8edc-15bdf6d0b18e admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:42.420002 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636b920b-6506-444e-8edc-15bdf6d0b18e admin admin] PATH_INFO: `/v3/users` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:42.420699 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636b920b-6506-444e-8edc-15bdf6d0b18e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:42.422749 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636b920b-6506-444e-8edc-15bdf6d0b18e admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:42.432078 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 936/1863] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:56:42 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:42.445366 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a51a85-f1c1-46fb-8c85-830340ee8f8c None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:42.447364 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-18a51a85-f1c1-46fb-8c85-830340ee8f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:42.461994 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:42.536708 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a51a85-f1c1-46fb-8c85-830340ee8f8c None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:42.540710 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36c9d53-91b7-4f03-85da-a50c71e63258 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbb9ac667f04c35a63b2689df18efed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:42.541343 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36c9d53-91b7-4f03-85da-a50c71e63258 service cinder] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:42.541617 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36c9d53-91b7-4f03-85da-a50c71e63258 service cinder] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:42.541891 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36c9d53-91b7-4f03-85da-a50c71e63258 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:42.552301 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36c9d53-91b7-4f03-85da-a50c71e63258 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:42.553838 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36c9d53-91b7-4f03-85da-a50c71e63258 service cinder] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:42.568255 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 937/1864] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:56:42 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:56:42.570817 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 928/1865] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 08:56:42 2022] POST /identity/v3/users => generated 281 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:56:42.571591 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36c9d53-91b7-4f03-85da-a50c71e63258 service cinder] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:42.571591 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-d36c9d53-91b7-4f03-85da-a50c71e63258 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:42.574673 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636b920b-6506-444e-8edc-15bdf6d0b18e admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:42.586410 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36c9d53-91b7-4f03-85da-a50c71e63258 service cinder] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:42.593077 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636b920b-6506-444e-8edc-15bdf6d0b18e admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:42.593275 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548e553f-1b10-4e33-9744-2a4e64bfbf09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:42.594100 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548e553f-1b10-4e33-9744-2a4e64bfbf09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:42.594596 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548e553f-1b10-4e33-9744-2a4e64bfbf09 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:42.595040 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548e553f-1b10-4e33-9744-2a4e64bfbf09 admin admin] PATH_INFO: `/v3/projects/fdc545d87c8344379548da5f65552dad` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:42.596376 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548e553f-1b10-4e33-9744-2a4e64bfbf09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdc545d87c8344379548da5f65552dad)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:42.604302 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548e553f-1b10-4e33-9744-2a4e64bfbf09 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:42.605216 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d85b165-7e23-4080-9e7e-50d051673004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:42.606260 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d85b165-7e23-4080-9e7e-50d051673004 admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:42.606760 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d85b165-7e23-4080-9e7e-50d051673004 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:42.607492 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d85b165-7e23-4080-9e7e-50d051673004 admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:42.608428 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d85b165-7e23-4080-9e7e-50d051673004 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:42.611722 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d85b165-7e23-4080-9e7e-50d051673004 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:42.632600 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 929/1866] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:56:42 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:56:42.637263 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d85b165-7e23-4080-9e7e-50d051673004 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:42.655963 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d85b165-7e23-4080-9e7e-50d051673004 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:42.668068 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce278a1-c7d9-4454-91ff-ca869ab6dade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:42.668068 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce278a1-c7d9-4454-91ff-ca869ab6dade admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:42.668068 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce278a1-c7d9-4454-91ff-ca869ab6dade admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:42.668068 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce278a1-c7d9-4454-91ff-ca869ab6dade admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:42.668527 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce278a1-c7d9-4454-91ff-ca869ab6dade admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:42.671040 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce278a1-c7d9-4454-91ff-ca869ab6dade admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:42.692530 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 930/1867] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:56:42 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:56:42.700420 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce278a1-c7d9-4454-91ff-ca869ab6dade admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:42.711981 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce278a1-c7d9-4454-91ff-ca869ab6dade admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:42.717475 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3bd953-9b83-4f63-b773-2acec5d985df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:42.718336 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3bd953-9b83-4f63-b773-2acec5d985df admin admin] REQUEST_METHOD: `PUT` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:42.718664 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3bd953-9b83-4f63-b773-2acec5d985df admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:42.718985 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3bd953-9b83-4f63-b773-2acec5d985df admin admin] PATH_INFO: `/v3/projects/0f84033eb3a749329d08b35ce92471a5/users/326386208c704b829955fcc1f56a402c/roles/b674fd0a592e4e0cada811bba5ceea4d` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:42.742086 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3bd953-9b83-4f63-b773-2acec5d985df admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f84033eb3a749329d08b35ce92471a5, user_id=326386208c704b829955fcc1f56a402c, role_id=b674fd0a592e4e0cada811bba5ceea4d)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:42.745250 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3bd953-9b83-4f63-b773-2acec5d985df admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:42.771633 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-548e553f-1b10-4e33-9744-2a4e64bfbf09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdc545d87c8344379548da5f65552dad', 'request_id': 'req-548e553f-1b10-4e33-9744-2a4e64bfbf09'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:42.777567 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-548e553f-1b10-4e33-9744-2a4e64bfbf09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdc545d87c8344379548da5f65552dad', 'request_id': 'req-548e553f-1b10-4e33-9744-2a4e64bfbf09'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:42.900450 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-548e553f-1b10-4e33-9744-2a4e64bfbf09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdc545d87c8344379548da5f65552dad', 'request_id': 'req-548e553f-1b10-4e33-9744-2a4e64bfbf09'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:42.925447 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 931/1868] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 08:56:42 2022] PUT /identity/v3/projects/0f84033eb3a749329d08b35ce92471a5/users/326386208c704b829955fcc1f56a402c/roles/b674fd0a592e4e0cada811bba5ceea4d => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:56:42.933317 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec16fbd-d38f-407d-8830-4d0913bb9c42 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:42.933773 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec16fbd-d38f-407d-8830-4d0913bb9c42 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:42.934248 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec16fbd-d38f-407d-8830-4d0913bb9c42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:42.940672 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-548e553f-1b10-4e33-9744-2a4e64bfbf09 admin admin] The token cache is being invalidate because project fdc545d87c8344379548da5f65552dad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:56:42.942537 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-548e553f-1b10-4e33-9744-2a4e64bfbf09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:42.942929 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-548e553f-1b10-4e33-9744-2a4e64bfbf09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:42.944930 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 938/1869] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 08:56:42 2022] DELETE /identity/v3/projects/fdc545d87c8344379548da5f65552dad => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:42.945638 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548e553f-1b10-4e33-9744-2a4e64bfbf09 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:42.947030 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-548e553f-1b10-4e33-9744-2a4e64bfbf09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:42.992481 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-8ec16fbd-d38f-407d-8830-4d0913bb9c42 None None] MFA Rules not processed for user `326386208c704b829955fcc1f56a402c`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:42.995019 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-8ec16fbd-d38f-407d-8830-4d0913bb9c42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:43.029288 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548e553f-1b10-4e33-9744-2a4e64bfbf09 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:43.088519 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 932/1870] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:56:42 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:56:43.090390 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c8e82b-5344-4274-8db3-a3dc9fdbbadd None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:43.090390 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c8e82b-5344-4274-8db3-a3dc9fdbbadd None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:43.090719 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c8e82b-5344-4274-8db3-a3dc9fdbbadd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:43.101642 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:43.111000 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50714f4e-8168-4d45-b3a6-9034eff403f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbb9ac667f04c35a63b2689df18efed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:43.122965 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50714f4e-8168-4d45-b3a6-9034eff403f9 service cinder] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:43.122965 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50714f4e-8168-4d45-b3a6-9034eff403f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:43.122965 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50714f4e-8168-4d45-b3a6-9034eff403f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:43.122965 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-50714f4e-8168-4d45-b3a6-9034eff403f9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:43.124499 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-c9c8e82b-5344-4274-8db3-a3dc9fdbbadd None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:43.124499 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-c9c8e82b-5344-4274-8db3-a3dc9fdbbadd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:43.216191 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50714f4e-8168-4d45-b3a6-9034eff403f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:43.217731 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50714f4e-8168-4d45-b3a6-9034eff403f9 service cinder] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:43.245006 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 939/1871] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:56:42 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:56:43.245935 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50714f4e-8168-4d45-b3a6-9034eff403f9 service cinder] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:43.247464 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-50714f4e-8168-4d45-b3a6-9034eff403f9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:43.277850 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50714f4e-8168-4d45-b3a6-9034eff403f9 service cinder] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:43.291768 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3964ef-3b93-4c32-bef7-28dcc09987c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbb9ac667f04c35a63b2689df18efed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:43.293046 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3964ef-3b93-4c32-bef7-28dcc09987c0 service cinder] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:43.293189 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3964ef-3b93-4c32-bef7-28dcc09987c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:43.293708 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3964ef-3b93-4c32-bef7-28dcc09987c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:43.309684 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 933/1872] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:56:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:43.312499 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5e2ae8-dc2e-4644-8ac4-8ce8aea4bd06 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:43.313342 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5e2ae8-dc2e-4644-8ac4-8ce8aea4bd06 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:43.313840 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5e2ae8-dc2e-4644-8ac4-8ce8aea4bd06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:43.322695 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3964ef-3b93-4c32-bef7-28dcc09987c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:43.325602 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3964ef-3b93-4c32-bef7-28dcc09987c0 service cinder] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:43.349650 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 940/1873] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:56:43 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:56:43.377060 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3964ef-3b93-4c32-bef7-28dcc09987c0 service cinder] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:43.379569 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-9d3964ef-3b93-4c32-bef7-28dcc09987c0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:43.393883 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-cb5e2ae8-dc2e-4644-8ac4-8ce8aea4bd06 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:43.399119 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-cb5e2ae8-dc2e-4644-8ac4-8ce8aea4bd06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:43.415018 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 934/1874] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:56:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:43.419767 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5e2ae8-dc2e-4644-8ac4-8ce8aea4bd06 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:43.429431 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3964ef-3b93-4c32-bef7-28dcc09987c0 service cinder] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:43.438376 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5e2ae8-dc2e-4644-8ac4-8ce8aea4bd06 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:43.446909 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1f0194-89ed-40a1-b15f-484a56f0e130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:43.448949 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1f0194-89ed-40a1-b15f-484a56f0e130 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:43.448949 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1f0194-89ed-40a1-b15f-484a56f0e130 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:43.448949 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1f0194-89ed-40a1-b15f-484a56f0e130 admin admin] PATH_INFO: `/v3/users/326386208c704b829955fcc1f56a402c` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:43.454294 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361fb736-1e9a-4d33-bab1-0924b81d2627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:43.454585 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1f0194-89ed-40a1-b15f-484a56f0e130 admin admin] RBAC: Authorizing `identity:delete_user(user_id=326386208c704b829955fcc1f56a402c)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:43.455027 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361fb736-1e9a-4d33-bab1-0924b81d2627 admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:43.455206 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361fb736-1e9a-4d33-bab1-0924b81d2627 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:43.455444 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361fb736-1e9a-4d33-bab1-0924b81d2627 admin admin] PATH_INFO: `/v3/domains` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:43.456111 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361fb736-1e9a-4d33-bab1-0924b81d2627 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:56:43.456418 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361fb736-1e9a-4d33-bab1-0924b81d2627 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:43.458642 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1f0194-89ed-40a1-b15f-484a56f0e130 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:43.461452 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361fb736-1e9a-4d33-bab1-0924b81d2627 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:43.471818 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:43.501285 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 935/1875] 10.222.0.55 () {58 vars in 1153 bytes} [Wed Aug 24 08:56:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:56:43.505633 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361fb736-1e9a-4d33-bab1-0924b81d2627 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:43.542507 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361fb736-1e9a-4d33-bab1-0924b81d2627 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:43.555386 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6799660-5e51-4784-9cd1-9d0116b3cea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:43.556529 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6799660-5e51-4784-9cd1-9d0116b3cea0 admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:43.558077 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6799660-5e51-4784-9cd1-9d0116b3cea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:43.558664 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6799660-5e51-4784-9cd1-9d0116b3cea0 admin admin] PATH_INFO: `/v3/projects` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:43.559909 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6799660-5e51-4784-9cd1-9d0116b3cea0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:43.565280 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6799660-5e51-4784-9cd1-9d0116b3cea0 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:43.711068 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 936/1876] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 08:56:43 2022] POST /identity/v3/projects => generated 385 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:56:43.716938 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6799660-5e51-4784-9cd1-9d0116b3cea0 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:43.786353 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-8c1f0194-89ed-40a1-b15f-484a56f0e130 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '326386208c704b829955fcc1f56a402c'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:43.788275 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-8c1f0194-89ed-40a1-b15f-484a56f0e130 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '326386208c704b829955fcc1f56a402c'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:43.789122 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-8c1f0194-89ed-40a1-b15f-484a56f0e130 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '326386208c704b829955fcc1f56a402c'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:43.810507 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-8c1f0194-89ed-40a1-b15f-484a56f0e130 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '326386208c704b829955fcc1f56a402c'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:43.842703 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6799660-5e51-4784-9cd1-9d0116b3cea0 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:43.849936 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178e4a0f-b741-4de8-83f0-d54703f030f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:43.850855 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178e4a0f-b741-4de8-83f0-d54703f030f0 admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:43.851299 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178e4a0f-b741-4de8-83f0-d54703f030f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:43.853802 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178e4a0f-b741-4de8-83f0-d54703f030f0 admin admin] PATH_INFO: `/v3/users` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:43.853802 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178e4a0f-b741-4de8-83f0-d54703f030f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:43.855207 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178e4a0f-b741-4de8-83f0-d54703f030f0 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:43.914536 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-8c1f0194-89ed-40a1-b15f-484a56f0e130 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '326386208c704b829955fcc1f56a402c'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:43.942075 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 941/1877] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 08:56:43 2022] DELETE /identity/v3/users/326386208c704b829955fcc1f56a402c => generated 0 bytes in 565 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:43.949349 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f833db7-d60d-46bd-9c6d-511ad1711a8d None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:43.949846 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f833db7-d60d-46bd-9c6d-511ad1711a8d None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:43.950329 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f833db7-d60d-46bd-9c6d-511ad1711a8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:43.985632 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-8f833db7-d60d-46bd-9c6d-511ad1711a8d None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:43.987622 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-8f833db7-d60d-46bd-9c6d-511ad1711a8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:44.004743 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 937/1878] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 08:56:43 2022] POST /identity/v3/users => generated 293 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:56:44.010683 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178e4a0f-b741-4de8-83f0-d54703f030f0 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:44.013063 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-178e4a0f-b741-4de8-83f0-d54703f030f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:44.037895 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178e4a0f-b741-4de8-83f0-d54703f030f0 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:44.137585 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 942/1879] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:56:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:56:44.151526 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f833db7-d60d-46bd-9c6d-511ad1711a8d None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:44.154048 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-8f833db7-d60d-46bd-9c6d-511ad1711a8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:44.161995 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4965bb8-4d83-4497-91de-a5005a636e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:44.163929 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4965bb8-4d83-4497-91de-a5005a636e27 admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:44.164462 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4965bb8-4d83-4497-91de-a5005a636e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:44.172431 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4965bb8-4d83-4497-91de-a5005a636e27 admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:44.172431 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4965bb8-4d83-4497-91de-a5005a636e27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:44.172928 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:44.173123 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4965bb8-4d83-4497-91de-a5005a636e27 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:44.190210 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 938/1880] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:56:44 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 08:56:44.197656 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4965bb8-4d83-4497-91de-a5005a636e27 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:44.233239 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4965bb8-4d83-4497-91de-a5005a636e27 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:44.257173 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aa37c7-a64a-471a-bd70-2ff6eb77605a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:44.258809 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa37c7-a64a-471a-bd70-2ff6eb77605a admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:44.259624 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa37c7-a64a-471a-bd70-2ff6eb77605a admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:44.260366 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa37c7-a64a-471a-bd70-2ff6eb77605a admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:44.261921 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aa37c7-a64a-471a-bd70-2ff6eb77605a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:44.272428 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aa37c7-a64a-471a-bd70-2ff6eb77605a admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:44.302145 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f833db7-d60d-46bd-9c6d-511ad1711a8d None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:44.304630 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 939/1881] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:56:44 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:56:44.309282 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b4a00d-8821-4ed6-affc-e35e4873ebea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:44.310283 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b4a00d-8821-4ed6-affc-e35e4873ebea service neutron] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:44.310749 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b4a00d-8821-4ed6-affc-e35e4873ebea service neutron] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:44.311076 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b4a00d-8821-4ed6-affc-e35e4873ebea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:44.314003 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aa37c7-a64a-471a-bd70-2ff6eb77605a admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:44.335336 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b4a00d-8821-4ed6-affc-e35e4873ebea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:44.338137 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b4a00d-8821-4ed6-affc-e35e4873ebea service neutron] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:44.349375 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aa37c7-a64a-471a-bd70-2ff6eb77605a admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:44.352438 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393b3e9b-07d7-43ff-b244-e78349856562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:44.353766 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393b3e9b-07d7-43ff-b244-e78349856562 admin admin] REQUEST_METHOD: `PUT` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:44.354233 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393b3e9b-07d7-43ff-b244-e78349856562 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:44.354710 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393b3e9b-07d7-43ff-b244-e78349856562 admin admin] PATH_INFO: `/v3/projects/70dc55b932d84222b5ca858d0322e210/users/396259d344d94f8188cbc74b60f971e7/roles/b674fd0a592e4e0cada811bba5ceea4d` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:44.358541 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 943/1882] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:56:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:56:44.380367 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393b3e9b-07d7-43ff-b244-e78349856562 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70dc55b932d84222b5ca858d0322e210, user_id=396259d344d94f8188cbc74b60f971e7, role_id=b674fd0a592e4e0cada811bba5ceea4d)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:44.393607 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393b3e9b-07d7-43ff-b244-e78349856562 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:44.555397 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 940/1883] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 08:56:44 2022] PUT /identity/v3/projects/70dc55b932d84222b5ca858d0322e210/users/396259d344d94f8188cbc74b60f971e7/roles/b674fd0a592e4e0cada811bba5ceea4d => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:56:44.567211 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d61523-4708-402d-a909-f4047c88ec76 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:44.567473 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d61523-4708-402d-a909-f4047c88ec76 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:44.567719 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d61523-4708-402d-a909-f4047c88ec76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:44.607801 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-44d61523-4708-402d-a909-f4047c88ec76 None None] MFA Rules not processed for user `396259d344d94f8188cbc74b60f971e7`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:44.609105 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-44d61523-4708-402d-a909-f4047c88ec76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:44.721072 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 944/1884] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:56:44 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:56:44.726880 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393b3e9b-07d7-43ff-b244-e78349856562 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:44.774528 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:44.849248 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393b3e9b-07d7-43ff-b244-e78349856562 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:44.849248 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f64f8ac-b8c4-49b3-b699-41fb21976ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:44.849248 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f64f8ac-b8c4-49b3-b699-41fb21976ff6 admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:44.849248 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f64f8ac-b8c4-49b3-b699-41fb21976ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:44.849801 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f64f8ac-b8c4-49b3-b699-41fb21976ff6 admin admin] PATH_INFO: `/v3/projects` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:44.849801 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f64f8ac-b8c4-49b3-b699-41fb21976ff6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:44.849801 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f64f8ac-b8c4-49b3-b699-41fb21976ff6 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:45.014391 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d61523-4708-402d-a909-f4047c88ec76 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:45.017023 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 941/1885] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 08:56:44 2022] POST /identity/v3/projects => generated 387 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:56:45.037847 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d61523-4708-402d-a909-f4047c88ec76 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:45.156908 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0b1060-2837-4230-8578-c8205b58c960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:45.157547 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0b1060-2837-4230-8578-c8205b58c960 admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:45.157835 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0b1060-2837-4230-8578-c8205b58c960 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:45.158107 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0b1060-2837-4230-8578-c8205b58c960 admin admin] PATH_INFO: `/v3/users` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:45.161074 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0b1060-2837-4230-8578-c8205b58c960 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:45.161074 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0b1060-2837-4230-8578-c8205b58c960 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:45.189832 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f64f8ac-b8c4-49b3-b699-41fb21976ff6 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:45.192312 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-8f64f8ac-b8c4-49b3-b699-41fb21976ff6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:45.207637 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f64f8ac-b8c4-49b3-b699-41fb21976ff6 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:45.213502 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabdc8a5-eb65-4f81-a0ea-babbd194a11d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:45.214170 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabdc8a5-eb65-4f81-a0ea-babbd194a11d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:45.214456 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabdc8a5-eb65-4f81-a0ea-babbd194a11d admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:45.214854 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabdc8a5-eb65-4f81-a0ea-babbd194a11d admin admin] PATH_INFO: `/v3/projects/0f84033eb3a749329d08b35ce92471a5` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:45.237506 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabdc8a5-eb65-4f81-a0ea-babbd194a11d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f84033eb3a749329d08b35ce92471a5)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:45.237506 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabdc8a5-eb65-4f81-a0ea-babbd194a11d admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:45.305544 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 945/1886] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 08:56:45 2022] POST /identity/v3/users => generated 300 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:56:45.310898 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0b1060-2837-4230-8578-c8205b58c960 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:45.326909 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0b1060-2837-4230-8578-c8205b58c960 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:45.334305 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25b9912-2c59-4f75-8f41-d5726de0736f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:45.334977 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25b9912-2c59-4f75-8f41-d5726de0736f admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:45.335266 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25b9912-2c59-4f75-8f41-d5726de0736f admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:45.335541 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25b9912-2c59-4f75-8f41-d5726de0736f admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:45.336293 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25b9912-2c59-4f75-8f41-d5726de0736f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:45.337909 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25b9912-2c59-4f75-8f41-d5726de0736f admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:45.349684 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 946/1887] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:56:45 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:56:45.356050 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25b9912-2c59-4f75-8f41-d5726de0736f admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:45.367602 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25b9912-2c59-4f75-8f41-d5726de0736f admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:45.370597 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-aabdc8a5-eb65-4f81-a0ea-babbd194a11d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f84033eb3a749329d08b35ce92471a5', 'request_id': 'req-aabdc8a5-eb65-4f81-a0ea-babbd194a11d'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:45.383329 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:45.459116 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaece43c-64df-48a4-860b-c6dd4bd6039d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:45.460277 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaece43c-64df-48a4-860b-c6dd4bd6039d admin admin] REQUEST_METHOD: `PUT` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:45.460818 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaece43c-64df-48a4-860b-c6dd4bd6039d admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:45.461316 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaece43c-64df-48a4-860b-c6dd4bd6039d admin admin] PATH_INFO: `/v3/projects/1d3eca9f28f24a938bd28159990bdf7c/users/99a0f93f18164b3482898132dfb8f142/roles/b870cd1acaa744e1ae1d58b9ac283821` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:45.497200 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaece43c-64df-48a4-860b-c6dd4bd6039d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d3eca9f28f24a938bd28159990bdf7c, user_id=99a0f93f18164b3482898132dfb8f142, role_id=b870cd1acaa744e1ae1d58b9ac283821)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:45.503354 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaece43c-64df-48a4-860b-c6dd4bd6039d admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:45.518918 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-aabdc8a5-eb65-4f81-a0ea-babbd194a11d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f84033eb3a749329d08b35ce92471a5', 'request_id': 'req-aabdc8a5-eb65-4f81-a0ea-babbd194a11d'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:45.539689 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-aabdc8a5-eb65-4f81-a0ea-babbd194a11d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f84033eb3a749329d08b35ce92471a5', 'request_id': 'req-aabdc8a5-eb65-4f81-a0ea-babbd194a11d'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:45.563503 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-aabdc8a5-eb65-4f81-a0ea-babbd194a11d admin admin] The token cache is being invalidate because project 0f84033eb3a749329d08b35ce92471a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:56:45.564109 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-aabdc8a5-eb65-4f81-a0ea-babbd194a11d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:45.565990 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-aabdc8a5-eb65-4f81-a0ea-babbd194a11d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:45.569094 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 942/1888] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 08:56:45 2022] DELETE /identity/v3/projects/0f84033eb3a749329d08b35ce92471a5 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:56:45.575171 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7f0969-f69e-49bf-a758-f2fbdc393841 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:45.576162 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7f0969-f69e-49bf-a758-f2fbdc393841 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:45.576975 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7f0969-f69e-49bf-a758-f2fbdc393841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:45.621898 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 947/1889] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 08:56:45 2022] PUT /identity/v3/projects/1d3eca9f28f24a938bd28159990bdf7c/users/99a0f93f18164b3482898132dfb8f142/roles/b870cd1acaa744e1ae1d58b9ac283821 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:45.632590 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaece43c-64df-48a4-860b-c6dd4bd6039d admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:45.635921 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-ea7f0969-f69e-49bf-a758-f2fbdc393841 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:45.638289 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-ea7f0969-f69e-49bf-a758-f2fbdc393841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:45.640856 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-aaece43c-64df-48a4-860b-c6dd4bd6039d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:45.799331 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 943/1890] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:56:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:56:45.808928 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05de043-dca2-48ec-a460-4b9906638103 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:45.809360 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05de043-dca2-48ec-a460-4b9906638103 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:45.809740 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05de043-dca2-48ec-a460-4b9906638103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:45.819526 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaece43c-64df-48a4-860b-c6dd4bd6039d admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:45.829822 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6ce0a5-493c-46b3-8854-62be73a004d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:45.836025 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6ce0a5-493c-46b3-8854-62be73a004d1 admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:45.836025 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6ce0a5-493c-46b3-8854-62be73a004d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:45.836025 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6ce0a5-493c-46b3-8854-62be73a004d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:45.836025 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6ce0a5-493c-46b3-8854-62be73a004d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:45.839388 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6ce0a5-493c-46b3-8854-62be73a004d1 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:45.842940 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-a05de043-dca2-48ec-a460-4b9906638103 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:45.845562 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-a05de043-dca2-48ec-a460-4b9906638103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:45.863006 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 944/1891] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:56:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:56:45.893907 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05de043-dca2-48ec-a460-4b9906638103 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:45.895236 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 948/1892] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:56:45 2022] GET /identity/v3/roles => generated 1420 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:56:45.898902 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6ce0a5-493c-46b3-8854-62be73a004d1 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:45.914113 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05de043-dca2-48ec-a460-4b9906638103 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:45.923767 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f968a812-4102-4b76-b289-4f0a364f1a7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:45.925007 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f968a812-4102-4b76-b289-4f0a364f1a7d admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:45.925490 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f968a812-4102-4b76-b289-4f0a364f1a7d admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:45.926168 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f968a812-4102-4b76-b289-4f0a364f1a7d admin admin] PATH_INFO: `/v3/domains` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:45.926750 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6ce0a5-493c-46b3-8854-62be73a004d1 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:45.927488 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f968a812-4102-4b76-b289-4f0a364f1a7d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:56:45.928011 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f968a812-4102-4b76-b289-4f0a364f1a7d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:45.933122 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f968a812-4102-4b76-b289-4f0a364f1a7d admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:45.937658 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e4e483-4dd7-4030-8f73-f2d6773aebb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:45.941830 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e4e483-4dd7-4030-8f73-f2d6773aebb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:45.944183 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e4e483-4dd7-4030-8f73-f2d6773aebb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:45.945489 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e4e483-4dd7-4030-8f73-f2d6773aebb5 admin admin] PATH_INFO: `/v3/projects/1d3eca9f28f24a938bd28159990bdf7c/users/99a0f93f18164b3482898132dfb8f142/roles/b870cd1acaa744e1ae1d58b9ac283821` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:45.950087 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e4e483-4dd7-4030-8f73-f2d6773aebb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d3eca9f28f24a938bd28159990bdf7c, user_id=99a0f93f18164b3482898132dfb8f142, role_id=b870cd1acaa744e1ae1d58b9ac283821)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:45.955840 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e4e483-4dd7-4030-8f73-f2d6773aebb5 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:45.976791 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 945/1893] 10.222.0.55 () {58 vars in 1153 bytes} [Wed Aug 24 08:56:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:56:45.979655 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 949/1894] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 08:56:45 2022] PUT /identity/v3/projects/1d3eca9f28f24a938bd28159990bdf7c/users/99a0f93f18164b3482898132dfb8f142/roles/b870cd1acaa744e1ae1d58b9ac283821 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:45.983308 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f968a812-4102-4b76-b289-4f0a364f1a7d admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:46.008310 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dfc89a-008a-459f-8ec0-7b8058b0af83 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:46.009007 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dfc89a-008a-459f-8ec0-7b8058b0af83 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:46.009515 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dfc89a-008a-459f-8ec0-7b8058b0af83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:46.047537 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f968a812-4102-4b76-b289-4f0a364f1a7d admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:46.071199 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:46.094603 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-42dfc89a-008a-459f-8ec0-7b8058b0af83 None None] MFA Rules not processed for user `99a0f93f18164b3482898132dfb8f142`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:46.098519 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-42dfc89a-008a-459f-8ec0-7b8058b0af83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:46.215186 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa73a94e-cce9-43ba-86d5-2d2e3b4f2d7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:46.215379 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa73a94e-cce9-43ba-86d5-2d2e3b4f2d7e admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:46.215867 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa73a94e-cce9-43ba-86d5-2d2e3b4f2d7e admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:46.216361 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa73a94e-cce9-43ba-86d5-2d2e3b4f2d7e admin admin] PATH_INFO: `/v3/projects` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:46.217873 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa73a94e-cce9-43ba-86d5-2d2e3b4f2d7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:46.221147 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa73a94e-cce9-43ba-86d5-2d2e3b4f2d7e admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:46.295077 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 950/1895] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:56:45 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:46.356856 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 946/1896] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 08:56:45 2022] POST /identity/v3/projects => generated 371 bytes in 374 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:56:46.358219 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dfc89a-008a-459f-8ec0-7b8058b0af83 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:46.378245 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:46.384318 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dfc89a-008a-459f-8ec0-7b8058b0af83 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:46.425090 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa73a94e-cce9-43ba-86d5-2d2e3b4f2d7e admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:46.430206 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-aa73a94e-cce9-43ba-86d5-2d2e3b4f2d7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:46.549705 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6729fd-3f43-4ded-b251-34859b443d9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:46.552792 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6729fd-3f43-4ded-b251-34859b443d9f admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:46.555614 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6729fd-3f43-4ded-b251-34859b443d9f admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:46.555614 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6729fd-3f43-4ded-b251-34859b443d9f admin admin] PATH_INFO: `/v3/users` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:46.556176 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6729fd-3f43-4ded-b251-34859b443d9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:46.562971 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6729fd-3f43-4ded-b251-34859b443d9f admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:46.578137 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa73a94e-cce9-43ba-86d5-2d2e3b4f2d7e admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:46.582837 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6074d2-8679-4f1c-81f7-b99721589db1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbb9ac667f04c35a63b2689df18efed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:46.583735 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6074d2-8679-4f1c-81f7-b99721589db1 service cinder] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:46.584139 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6074d2-8679-4f1c-81f7-b99721589db1 service cinder] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:46.584532 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6074d2-8679-4f1c-81f7-b99721589db1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:46.697789 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 951/1897] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 08:56:46 2022] POST /identity/v3/users => generated 286 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:56:46.702017 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6729fd-3f43-4ded-b251-34859b443d9f admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:46.717832 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6074d2-8679-4f1c-81f7-b99721589db1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:46.721813 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6074d2-8679-4f1c-81f7-b99721589db1 service cinder] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:46.728390 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6729fd-3f43-4ded-b251-34859b443d9f admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:46.747043 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f56b688-c274-48ca-8f9c-9ca24921cdf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:46.750428 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 947/1898] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:56:46 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:56:46.750738 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f56b688-c274-48ca-8f9c-9ca24921cdf9 admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:46.751146 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f56b688-c274-48ca-8f9c-9ca24921cdf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:46.751635 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f56b688-c274-48ca-8f9c-9ca24921cdf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:46.752860 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f56b688-c274-48ca-8f9c-9ca24921cdf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:46.756699 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f56b688-c274-48ca-8f9c-9ca24921cdf9 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:46.777736 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 952/1899] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:56:46 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:56:46.791965 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6074d2-8679-4f1c-81f7-b99721589db1 service cinder] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:46.817175 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6074d2-8679-4f1c-81f7-b99721589db1 service cinder] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:46.828478 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f44632-cbe8-4b8a-9100-4143f876684c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:46.829821 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f44632-cbe8-4b8a-9100-4143f876684c admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:46.831492 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f44632-cbe8-4b8a-9100-4143f876684c admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:46.831492 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f44632-cbe8-4b8a-9100-4143f876684c admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:46.832560 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f44632-cbe8-4b8a-9100-4143f876684c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:46.840049 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f44632-cbe8-4b8a-9100-4143f876684c admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:46.863688 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 948/1900] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:56:46 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:56:46.872580 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f56b688-c274-48ca-8f9c-9ca24921cdf9 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:46.902235 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f56b688-c274-48ca-8f9c-9ca24921cdf9 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:46.908970 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0862cb6-8432-4a1f-9033-c087adce047a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:46.921035 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0862cb6-8432-4a1f-9033-c087adce047a admin admin] REQUEST_METHOD: `PUT` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:46.921035 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0862cb6-8432-4a1f-9033-c087adce047a admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:46.921035 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0862cb6-8432-4a1f-9033-c087adce047a admin admin] PATH_INFO: `/v3/projects/f9753d09aeea45378d637bf28536bb7d/users/bb815fa9f45a4b1db1f30fbfaeb3c4c9/roles/b674fd0a592e4e0cada811bba5ceea4d` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:46.942853 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0862cb6-8432-4a1f-9033-c087adce047a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9753d09aeea45378d637bf28536bb7d, user_id=bb815fa9f45a4b1db1f30fbfaeb3c4c9, role_id=b674fd0a592e4e0cada811bba5ceea4d)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:46.946168 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0862cb6-8432-4a1f-9033-c087adce047a admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:46.996901 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f44632-cbe8-4b8a-9100-4143f876684c admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:47.022775 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f44632-cbe8-4b8a-9100-4143f876684c admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:47.032683 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3a7e35-6553-48d8-a61f-3ea4a573c80b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:47.033765 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3a7e35-6553-48d8-a61f-3ea4a573c80b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:47.034491 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3a7e35-6553-48d8-a61f-3ea4a573c80b admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:47.035019 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3a7e35-6553-48d8-a61f-3ea4a573c80b admin admin] PATH_INFO: `/v3/users/396259d344d94f8188cbc74b60f971e7` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:47.037774 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 953/1901] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 08:56:46 2022] PUT /identity/v3/projects/f9753d09aeea45378d637bf28536bb7d/users/bb815fa9f45a4b1db1f30fbfaeb3c4c9/roles/b674fd0a592e4e0cada811bba5ceea4d => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:47.041016 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3a7e35-6553-48d8-a61f-3ea4a573c80b admin admin] RBAC: Authorizing `identity:delete_user(user_id=396259d344d94f8188cbc74b60f971e7)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:47.044432 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3a7e35-6553-48d8-a61f-3ea4a573c80b admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:47.046543 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5246c0ae-ebe6-4b03-b138-d7a9684e66d9 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:47.046543 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5246c0ae-ebe6-4b03-b138-d7a9684e66d9 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:47.046543 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5246c0ae-ebe6-4b03-b138-d7a9684e66d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:47.068110 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-5246c0ae-ebe6-4b03-b138-d7a9684e66d9 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:47.069489 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-5246c0ae-ebe6-4b03-b138-d7a9684e66d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:47.151422 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 954/1902] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:56:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:56:47.163597 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5246c0ae-ebe6-4b03-b138-d7a9684e66d9 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:47.166047 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-5246c0ae-ebe6-4b03-b138-d7a9684e66d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:47.247450 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5246c0ae-ebe6-4b03-b138-d7a9684e66d9 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:47.251332 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b67cde5-d88a-4284-ad7e-adf6f707c6b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:47.251962 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b67cde5-d88a-4284-ad7e-adf6f707c6b8 service neutron] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:47.252246 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b67cde5-d88a-4284-ad7e-adf6f707c6b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:47.252521 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b67cde5-d88a-4284-ad7e-adf6f707c6b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:47.264107 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b67cde5-d88a-4284-ad7e-adf6f707c6b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:47.266048 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b67cde5-d88a-4284-ad7e-adf6f707c6b8 service neutron] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:47.282233 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 955/1903] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:56:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:56:47.340142 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:47.364008 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-df3a7e35-6553-48d8-a61f-3ea4a573c80b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '396259d344d94f8188cbc74b60f971e7'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:47.365225 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-df3a7e35-6553-48d8-a61f-3ea4a573c80b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '396259d344d94f8188cbc74b60f971e7'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:47.366235 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-df3a7e35-6553-48d8-a61f-3ea4a573c80b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '396259d344d94f8188cbc74b60f971e7'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:47.396115 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-df3a7e35-6553-48d8-a61f-3ea4a573c80b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '396259d344d94f8188cbc74b60f971e7'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:47.511811 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-df3a7e35-6553-48d8-a61f-3ea4a573c80b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '396259d344d94f8188cbc74b60f971e7'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:47.532626 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 949/1904] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 08:56:46 2022] DELETE /identity/v3/users/396259d344d94f8188cbc74b60f971e7 => generated 0 bytes in 537 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:47.538676 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddb8fac-1280-4422-bf6d-ae5ff0886fc1 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:47.539479 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddb8fac-1280-4422-bf6d-ae5ff0886fc1 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:47.539479 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddb8fac-1280-4422-bf6d-ae5ff0886fc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:47.550464 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:47.582560 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-0ddb8fac-1280-4422-bf6d-ae5ff0886fc1 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:47.584085 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-0ddb8fac-1280-4422-bf6d-ae5ff0886fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:47.698894 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 956/1905] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:56:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:47.714030 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3a7e35-6553-48d8-a61f-3ea4a573c80b admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:47.716417 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-df3a7e35-6553-48d8-a61f-3ea4a573c80b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:47.825844 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3a7e35-6553-48d8-a61f-3ea4a573c80b admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:47.838794 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfb3cf0-9175-48e8-a631-7540a2d739af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:47.842120 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfb3cf0-9175-48e8-a631-7540a2d739af service neutron] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:47.844420 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfb3cf0-9175-48e8-a631-7540a2d739af service neutron] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:47.845786 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfb3cf0-9175-48e8-a631-7540a2d739af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:47.894626 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfb3cf0-9175-48e8-a631-7540a2d739af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:47.923304 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfb3cf0-9175-48e8-a631-7540a2d739af service neutron] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:47.939141 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 950/1906] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:56:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:56:48.907565 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddb8fac-1280-4422-bf6d-ae5ff0886fc1 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:48.909137 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-0ddb8fac-1280-4422-bf6d-ae5ff0886fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:48.923981 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddb8fac-1280-4422-bf6d-ae5ff0886fc1 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:48.932720 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7ff545-b615-4134-b4cf-d5883c3b2097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:48.932720 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7ff545-b615-4134-b4cf-d5883c3b2097 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:48.932720 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7ff545-b615-4134-b4cf-d5883c3b2097 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:48.932720 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7ff545-b615-4134-b4cf-d5883c3b2097 admin admin] PATH_INFO: `/v3/projects/70dc55b932d84222b5ca858d0322e210` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:48.934287 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7ff545-b615-4134-b4cf-d5883c3b2097 admin admin] RBAC: Authorizing `identity:delete_project(project_id=70dc55b932d84222b5ca858d0322e210)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:48.945480 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7ff545-b615-4134-b4cf-d5883c3b2097 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:49.190501 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-8d7ff545-b615-4134-b4cf-d5883c3b2097 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70dc55b932d84222b5ca858d0322e210', 'request_id': 'req-8d7ff545-b615-4134-b4cf-d5883c3b2097'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:49.210591 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-8d7ff545-b615-4134-b4cf-d5883c3b2097 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70dc55b932d84222b5ca858d0322e210', 'request_id': 'req-8d7ff545-b615-4134-b4cf-d5883c3b2097'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:49.305844 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-8d7ff545-b615-4134-b4cf-d5883c3b2097 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70dc55b932d84222b5ca858d0322e210', 'request_id': 'req-8d7ff545-b615-4134-b4cf-d5883c3b2097'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:49.334779 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-8d7ff545-b615-4134-b4cf-d5883c3b2097 admin admin] The token cache is being invalidate because project 70dc55b932d84222b5ca858d0322e210 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:56:49.336039 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-8d7ff545-b615-4134-b4cf-d5883c3b2097 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:49.336640 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-8d7ff545-b615-4134-b4cf-d5883c3b2097 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:49.340323 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 957/1907] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 08:56:48 2022] DELETE /identity/v3/projects/70dc55b932d84222b5ca858d0322e210 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:56:49.346893 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfb3cf0-9175-48e8-a631-7540a2d739af service neutron] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:49.369992 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfb3cf0-9175-48e8-a631-7540a2d739af service neutron] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:49.516048 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3d589b-40fb-4fc8-8f56-f76b05beb2a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:49.516787 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3d589b-40fb-4fc8-8f56-f76b05beb2a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:49.517068 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3d589b-40fb-4fc8-8f56-f76b05beb2a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:49.517351 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3d589b-40fb-4fc8-8f56-f76b05beb2a5 admin admin] PATH_INFO: `/v3/users/99a0f93f18164b3482898132dfb8f142` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:49.521290 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3d589b-40fb-4fc8-8f56-f76b05beb2a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=99a0f93f18164b3482898132dfb8f142)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:49.527111 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3d589b-40fb-4fc8-8f56-f76b05beb2a5 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:49.851778 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-ef3d589b-40fb-4fc8-8f56-f76b05beb2a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99a0f93f18164b3482898132dfb8f142'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:49.852509 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-ef3d589b-40fb-4fc8-8f56-f76b05beb2a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99a0f93f18164b3482898132dfb8f142'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:49.853107 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-ef3d589b-40fb-4fc8-8f56-f76b05beb2a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99a0f93f18164b3482898132dfb8f142'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:49.862089 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:49.874087 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-ef3d589b-40fb-4fc8-8f56-f76b05beb2a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99a0f93f18164b3482898132dfb8f142'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:50.003017 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-ef3d589b-40fb-4fc8-8f56-f76b05beb2a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99a0f93f18164b3482898132dfb8f142'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:50.061167 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 951/1908] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 08:56:49 2022] DELETE /identity/v3/users/99a0f93f18164b3482898132dfb8f142 => generated 0 bytes in 710 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:50.170871 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0bb487-0ba7-43b4-aa0a-a4b899df1b39 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:50.171825 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0bb487-0ba7-43b4-aa0a-a4b899df1b39 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:50.171825 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0bb487-0ba7-43b4-aa0a-a4b899df1b39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:50.266713 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-5e0bb487-0ba7-43b4-aa0a-a4b899df1b39 None None] MFA Rules not processed for user `bb815fa9f45a4b1db1f30fbfaeb3c4c9`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:50.268322 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-5e0bb487-0ba7-43b4-aa0a-a4b899df1b39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:50.400370 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 958/1909] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:56:50 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:50.412550 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3d589b-40fb-4fc8-8f56-f76b05beb2a5 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:50.414839 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-ef3d589b-40fb-4fc8-8f56-f76b05beb2a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:50.549524 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3d589b-40fb-4fc8-8f56-f76b05beb2a5 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:50.554886 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6047f1-d242-4955-b81a-76e59cad6ac6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbb9ac667f04c35a63b2689df18efed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:50.555919 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6047f1-d242-4955-b81a-76e59cad6ac6 service cinder] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:50.556415 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6047f1-d242-4955-b81a-76e59cad6ac6 service cinder] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:50.557120 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6047f1-d242-4955-b81a-76e59cad6ac6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:50.569269 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6047f1-d242-4955-b81a-76e59cad6ac6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:50.571952 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6047f1-d242-4955-b81a-76e59cad6ac6 service cinder] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:50.587348 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 952/1910] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:56:50 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:56:51.054622 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0bb487-0ba7-43b4-aa0a-a4b899df1b39 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:51.056846 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-5e0bb487-0ba7-43b4-aa0a-a4b899df1b39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:51.087627 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:51.179106 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0bb487-0ba7-43b4-aa0a-a4b899df1b39 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:51.185712 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308297b1-e181-44a2-b9cd-f3071025e37e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:51.186856 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308297b1-e181-44a2-b9cd-f3071025e37e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:51.187208 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308297b1-e181-44a2-b9cd-f3071025e37e admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:51.187646 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308297b1-e181-44a2-b9cd-f3071025e37e admin admin] PATH_INFO: `/v3/projects/1d3eca9f28f24a938bd28159990bdf7c` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:51.189338 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308297b1-e181-44a2-b9cd-f3071025e37e admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d3eca9f28f24a938bd28159990bdf7c)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:51.191399 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308297b1-e181-44a2-b9cd-f3071025e37e admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:51.457949 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-308297b1-e181-44a2-b9cd-f3071025e37e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d3eca9f28f24a938bd28159990bdf7c', 'request_id': 'req-308297b1-e181-44a2-b9cd-f3071025e37e'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:51.469932 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-308297b1-e181-44a2-b9cd-f3071025e37e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d3eca9f28f24a938bd28159990bdf7c', 'request_id': 'req-308297b1-e181-44a2-b9cd-f3071025e37e'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:51.589213 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-308297b1-e181-44a2-b9cd-f3071025e37e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d3eca9f28f24a938bd28159990bdf7c', 'request_id': 'req-308297b1-e181-44a2-b9cd-f3071025e37e'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:51.655277 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6047f1-d242-4955-b81a-76e59cad6ac6 service cinder] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:51.657046 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-8c6047f1-d242-4955-b81a-76e59cad6ac6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:51.661123 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-308297b1-e181-44a2-b9cd-f3071025e37e admin admin] The token cache is being invalidate because project 1d3eca9f28f24a938bd28159990bdf7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:56:51.667416 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-308297b1-e181-44a2-b9cd-f3071025e37e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:51.670555 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-308297b1-e181-44a2-b9cd-f3071025e37e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:51.676886 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 959/1911] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 08:56:51 2022] DELETE /identity/v3/projects/1d3eca9f28f24a938bd28159990bdf7c => generated 0 bytes in 622 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:56:51.679835 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdb31c8-cc98-4503-a3fa-f9f33640af44 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:51.680299 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdb31c8-cc98-4503-a3fa-f9f33640af44 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:51.684722 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdb31c8-cc98-4503-a3fa-f9f33640af44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:51.694663 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6047f1-d242-4955-b81a-76e59cad6ac6 service cinder] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:51.724244 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-7cdb31c8-cc98-4503-a3fa-f9f33640af44 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:51.725048 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-7cdb31c8-cc98-4503-a3fa-f9f33640af44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:51.774158 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3a3944-4326-47b3-9148-9a36b19992cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:51.774815 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3a3944-4326-47b3-9148-9a36b19992cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:51.775098 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3a3944-4326-47b3-9148-9a36b19992cb admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:51.775433 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3a3944-4326-47b3-9148-9a36b19992cb admin admin] PATH_INFO: `/v3/users/7c21c842aa3f4fcdab49136642a725c9` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:51.776683 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3a3944-4326-47b3-9148-9a36b19992cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c21c842aa3f4fcdab49136642a725c9)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:51.779021 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3a3944-4326-47b3-9148-9a36b19992cb admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:51.845220 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 960/1912] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:56:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:51.861022 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838d66fc-6e69-4f32-8f8e-c1fa2fbb8820 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:51.861593 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838d66fc-6e69-4f32-8f8e-c1fa2fbb8820 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:51.863988 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838d66fc-6e69-4f32-8f8e-c1fa2fbb8820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:51.897240 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-838d66fc-6e69-4f32-8f8e-c1fa2fbb8820 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:51.898526 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-838d66fc-6e69-4f32-8f8e-c1fa2fbb8820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:51.907070 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 961/1913] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:56:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:56:51.918296 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838d66fc-6e69-4f32-8f8e-c1fa2fbb8820 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:51.929899 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838d66fc-6e69-4f32-8f8e-c1fa2fbb8820 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:51.935912 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0f3686-4cc1-47ac-993f-3af487b340b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:51.936678 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0f3686-4cc1-47ac-993f-3af487b340b0 admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:51.937012 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0f3686-4cc1-47ac-993f-3af487b340b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:51.937387 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0f3686-4cc1-47ac-993f-3af487b340b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:51.938134 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0f3686-4cc1-47ac-993f-3af487b340b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:56:51.938513 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0f3686-4cc1-47ac-993f-3af487b340b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:51.940506 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0f3686-4cc1-47ac-993f-3af487b340b0 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:51.959041 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 962/1914] 10.222.0.55 () {58 vars in 1153 bytes} [Wed Aug 24 08:56:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:56:51.964226 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0f3686-4cc1-47ac-993f-3af487b340b0 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:51.981892 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0f3686-4cc1-47ac-993f-3af487b340b0 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:51.996701 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f1079f-7ca1-445e-8638-ae1486256487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:51.997663 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f1079f-7ca1-445e-8638-ae1486256487 admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:51.997663 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f1079f-7ca1-445e-8638-ae1486256487 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:51.998320 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f1079f-7ca1-445e-8638-ae1486256487 admin admin] PATH_INFO: `/v3/projects` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:52.001602 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f1079f-7ca1-445e-8638-ae1486256487 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:52.008226 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f1079f-7ca1-445e-8638-ae1486256487 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:52.042836 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-9c3a3944-4326-47b3-9148-9a36b19992cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c21c842aa3f4fcdab49136642a725c9'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:52.044974 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-9c3a3944-4326-47b3-9148-9a36b19992cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c21c842aa3f4fcdab49136642a725c9'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:52.046168 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-9c3a3944-4326-47b3-9148-9a36b19992cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c21c842aa3f4fcdab49136642a725c9'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:52.074059 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-9c3a3944-4326-47b3-9148-9a36b19992cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c21c842aa3f4fcdab49136642a725c9'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:52.183044 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 963/1915] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 08:56:51 2022] POST /identity/v3/projects => generated 385 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:56:52.187824 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f1079f-7ca1-445e-8638-ae1486256487 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:52.190813 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-99f1079f-7ca1-445e-8638-ae1486256487 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:52.217763 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-9c3a3944-4326-47b3-9148-9a36b19992cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c21c842aa3f4fcdab49136642a725c9'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:52.223096 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:52.245596 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 953/1916] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 08:56:51 2022] DELETE /identity/v3/users/7c21c842aa3f4fcdab49136642a725c9 => generated 0 bytes in 591 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:52.251342 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e25b03e-1d0c-40d2-8304-1eb3fabfccfe None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:52.251927 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e25b03e-1d0c-40d2-8304-1eb3fabfccfe None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:52.252399 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e25b03e-1d0c-40d2-8304-1eb3fabfccfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:52.277302 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-8e25b03e-1d0c-40d2-8304-1eb3fabfccfe None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:52.279253 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-8e25b03e-1d0c-40d2-8304-1eb3fabfccfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:52.348525 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f1079f-7ca1-445e-8638-ae1486256487 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:52.375945 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:52.401078 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 954/1917] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:56:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:52.421676 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e25b03e-1d0c-40d2-8304-1eb3fabfccfe None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:52.423260 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-8e25b03e-1d0c-40d2-8304-1eb3fabfccfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:52.471349 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefd567e-ca1f-4455-baa5-393037236a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:52.472458 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefd567e-ca1f-4455-baa5-393037236a83 admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:52.472963 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefd567e-ca1f-4455-baa5-393037236a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:52.473557 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefd567e-ca1f-4455-baa5-393037236a83 admin admin] PATH_INFO: `/v3/users` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:52.474493 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefd567e-ca1f-4455-baa5-393037236a83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:52.478606 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefd567e-ca1f-4455-baa5-393037236a83 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:52.556148 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e25b03e-1d0c-40d2-8304-1eb3fabfccfe None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:52.561844 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f9091e-890f-4963-8284-293bf246d1b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:52.562694 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f9091e-890f-4963-8284-293bf246d1b3 service neutron] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:52.563177 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f9091e-890f-4963-8284-293bf246d1b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:52.563622 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f9091e-890f-4963-8284-293bf246d1b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:52.578143 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f9091e-890f-4963-8284-293bf246d1b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:52.580828 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f9091e-890f-4963-8284-293bf246d1b3 service neutron] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:52.599722 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 955/1918] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:56:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:56:52.620247 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 964/1919] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 08:56:52 2022] POST /identity/v3/users => generated 293 bytes in 434 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:56:52.626912 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f9091e-890f-4963-8284-293bf246d1b3 service neutron] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:52.640230 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f9091e-890f-4963-8284-293bf246d1b3 service neutron] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:52.646895 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bf6eae-99a5-4d8f-943f-6203d56ba694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:52.647659 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bf6eae-99a5-4d8f-943f-6203d56ba694 admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:52.648031 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bf6eae-99a5-4d8f-943f-6203d56ba694 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:52.648396 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bf6eae-99a5-4d8f-943f-6203d56ba694 admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:52.649598 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bf6eae-99a5-4d8f-943f-6203d56ba694 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:52.651745 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bf6eae-99a5-4d8f-943f-6203d56ba694 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:52.665458 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 956/1920] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:56:52 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:56:52.670543 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefd567e-ca1f-4455-baa5-393037236a83 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:52.691944 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefd567e-ca1f-4455-baa5-393037236a83 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:52.700035 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89601bfb-e917-4ea8-920e-46c89253030e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:52.701278 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89601bfb-e917-4ea8-920e-46c89253030e admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:52.701760 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89601bfb-e917-4ea8-920e-46c89253030e admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:52.702208 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89601bfb-e917-4ea8-920e-46c89253030e admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:52.703391 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89601bfb-e917-4ea8-920e-46c89253030e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:52.706074 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89601bfb-e917-4ea8-920e-46c89253030e admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:52.720613 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 965/1921] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:56:52 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:56:52.726463 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bf6eae-99a5-4d8f-943f-6203d56ba694 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:52.740734 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bf6eae-99a5-4d8f-943f-6203d56ba694 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:52.751897 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc222cb-7fa8-4b25-9207-62248e15e3ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:52.752856 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc222cb-7fa8-4b25-9207-62248e15e3ac admin admin] REQUEST_METHOD: `PUT` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:52.753421 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc222cb-7fa8-4b25-9207-62248e15e3ac admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:52.754668 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc222cb-7fa8-4b25-9207-62248e15e3ac admin admin] PATH_INFO: `/v3/projects/b503eeea4d854d30a5107e6d65048fb8/users/66863e52e6d94b159814fa4426394628/roles/b674fd0a592e4e0cada811bba5ceea4d` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:52.783434 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc222cb-7fa8-4b25-9207-62248e15e3ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=b503eeea4d854d30a5107e6d65048fb8, user_id=66863e52e6d94b159814fa4426394628, role_id=b674fd0a592e4e0cada811bba5ceea4d)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:52.789078 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc222cb-7fa8-4b25-9207-62248e15e3ac admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:52.986906 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 957/1922] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 08:56:52 2022] PUT /identity/v3/projects/b503eeea4d854d30a5107e6d65048fb8/users/66863e52e6d94b159814fa4426394628/roles/b674fd0a592e4e0cada811bba5ceea4d => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:52.999253 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550caf82-48e7-4d5c-a422-a5befa6d328a None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:52.999680 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550caf82-48e7-4d5c-a422-a5befa6d328a None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:53.001091 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550caf82-48e7-4d5c-a422-a5befa6d328a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:53.068231 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-550caf82-48e7-4d5c-a422-a5befa6d328a None None] MFA Rules not processed for user `66863e52e6d94b159814fa4426394628`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:53.070579 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-550caf82-48e7-4d5c-a422-a5befa6d328a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:53.199925 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 966/1923] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:56:52 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:56:53.204669 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc222cb-7fa8-4b25-9207-62248e15e3ac admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:53.407819 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc222cb-7fa8-4b25-9207-62248e15e3ac admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:53.432227 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047eec4a-0db8-4f9a-8af2-3311d5801a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:53.434733 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047eec4a-0db8-4f9a-8af2-3311d5801a0e admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:53.436374 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047eec4a-0db8-4f9a-8af2-3311d5801a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:53.438707 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047eec4a-0db8-4f9a-8af2-3311d5801a0e admin admin] PATH_INFO: `/v3/projects` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:53.439929 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047eec4a-0db8-4f9a-8af2-3311d5801a0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:53.449595 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047eec4a-0db8-4f9a-8af2-3311d5801a0e admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:53.491572 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:53.628666 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 958/1924] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 08:56:53 2022] POST /identity/v3/projects => generated 385 bytes in 421 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:56:53.632746 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550caf82-48e7-4d5c-a422-a5befa6d328a None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:53.651090 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550caf82-48e7-4d5c-a422-a5befa6d328a None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:53.682735 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:53.763685 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047eec4a-0db8-4f9a-8af2-3311d5801a0e admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:53.768718 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-047eec4a-0db8-4f9a-8af2-3311d5801a0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:53.777033 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3659900b-debe-4c26-bc2d-faa91aa245f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:53.777783 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3659900b-debe-4c26-bc2d-faa91aa245f8 admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:53.778847 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3659900b-debe-4c26-bc2d-faa91aa245f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:53.779352 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3659900b-debe-4c26-bc2d-faa91aa245f8 admin admin] PATH_INFO: `/v3/users` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:53.780187 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3659900b-debe-4c26-bc2d-faa91aa245f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:53.783403 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3659900b-debe-4c26-bc2d-faa91aa245f8 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:53.798923 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047eec4a-0db8-4f9a-8af2-3311d5801a0e admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:53.816010 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651e88ad-69e9-46d0-9180-3b8b26bfa8cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:53.820292 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651e88ad-69e9-46d0-9180-3b8b26bfa8cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:53.820643 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651e88ad-69e9-46d0-9180-3b8b26bfa8cf admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:53.821802 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651e88ad-69e9-46d0-9180-3b8b26bfa8cf admin admin] PATH_INFO: `/v3/projects/b3c25dd5adae4846b4bbac08ad2d9a44` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:53.823336 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651e88ad-69e9-46d0-9180-3b8b26bfa8cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3c25dd5adae4846b4bbac08ad2d9a44)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:53.827750 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651e88ad-69e9-46d0-9180-3b8b26bfa8cf admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:53.916338 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 967/1925] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 08:56:53 2022] POST /identity/v3/users => generated 299 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:56:53.917705 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced79f0-656f-4286-b926-f50b7736e4ad None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:53.918022 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced79f0-656f-4286-b926-f50b7736e4ad None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:53.918312 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced79f0-656f-4286-b926-f50b7736e4ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:53.945604 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-8ced79f0-656f-4286-b926-f50b7736e4ad None None] MFA Rules not processed for user `bb815fa9f45a4b1db1f30fbfaeb3c4c9`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:53.947158 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-8ced79f0-656f-4286-b926-f50b7736e4ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:53.997284 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-651e88ad-69e9-46d0-9180-3b8b26bfa8cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3c25dd5adae4846b4bbac08ad2d9a44', 'request_id': 'req-651e88ad-69e9-46d0-9180-3b8b26bfa8cf'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:54.064628 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 968/1926] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:56:53 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:54.065613 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ced79f0-656f-4286-b926-f50b7736e4ad None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:54.079448 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-651e88ad-69e9-46d0-9180-3b8b26bfa8cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3c25dd5adae4846b4bbac08ad2d9a44', 'request_id': 'req-651e88ad-69e9-46d0-9180-3b8b26bfa8cf'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:54.088386 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-651e88ad-69e9-46d0-9180-3b8b26bfa8cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3c25dd5adae4846b4bbac08ad2d9a44', 'request_id': 'req-651e88ad-69e9-46d0-9180-3b8b26bfa8cf'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:54.091130 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ced79f0-656f-4286-b926-f50b7736e4ad None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:54.105535 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-651e88ad-69e9-46d0-9180-3b8b26bfa8cf admin admin] The token cache is being invalidate because project b3c25dd5adae4846b4bbac08ad2d9a44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:56:54.105974 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-651e88ad-69e9-46d0-9180-3b8b26bfa8cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:54.109218 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-651e88ad-69e9-46d0-9180-3b8b26bfa8cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:54.110634 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 959/1927] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 08:56:53 2022] DELETE /identity/v3/projects/b3c25dd5adae4846b4bbac08ad2d9a44 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:56:54.118775 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651e88ad-69e9-46d0-9180-3b8b26bfa8cf admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:54.128979 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-651e88ad-69e9-46d0-9180-3b8b26bfa8cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:54.272469 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f22a69-c3d7-4530-b84c-b6bddf9cf1ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:54.274775 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f22a69-c3d7-4530-b84c-b6bddf9cf1ba admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:54.275484 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f22a69-c3d7-4530-b84c-b6bddf9cf1ba admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:54.276521 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f22a69-c3d7-4530-b84c-b6bddf9cf1ba admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:54.278259 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f22a69-c3d7-4530-b84c-b6bddf9cf1ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:54.284138 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f22a69-c3d7-4530-b84c-b6bddf9cf1ba admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:54.291682 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651e88ad-69e9-46d0-9180-3b8b26bfa8cf admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:54.302741 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42402f8d-02ad-485c-a22c-196a234ef913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:54.304226 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42402f8d-02ad-485c-a22c-196a234ef913 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:54.304816 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42402f8d-02ad-485c-a22c-196a234ef913 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:54.305376 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42402f8d-02ad-485c-a22c-196a234ef913 admin admin] PATH_INFO: `/v3/users/f8238e09838d47d1bd59e0e7e7a0fab2` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:54.308245 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42402f8d-02ad-485c-a22c-196a234ef913 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8238e09838d47d1bd59e0e7e7a0fab2)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:54.310161 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 969/1928] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:56:54 2022] GET /identity/v3/roles => generated 1420 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:56:54.311520 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42402f8d-02ad-485c-a22c-196a234ef913 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:54.314828 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f22a69-c3d7-4530-b84c-b6bddf9cf1ba admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:54.316737 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-c3f22a69-c3d7-4530-b84c-b6bddf9cf1ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:54.333289 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f22a69-c3d7-4530-b84c-b6bddf9cf1ba admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:54.346706 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3a0587-ef4f-492a-886c-e973994f0bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:54.347419 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3a0587-ef4f-492a-886c-e973994f0bdc admin admin] REQUEST_METHOD: `PUT` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:54.348253 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3a0587-ef4f-492a-886c-e973994f0bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:54.348762 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3a0587-ef4f-492a-886c-e973994f0bdc admin admin] PATH_INFO: `/v3/projects/27be6cbbc389495ca1f74ef2b5e8a535/users/9cd14f6027a2471e8eb79d0e434f7e2b/roles/b870cd1acaa744e1ae1d58b9ac283821` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:54.367487 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3a0587-ef4f-492a-886c-e973994f0bdc admin admin] RBAC: Authorizing `identity:create_grant(project_id=27be6cbbc389495ca1f74ef2b5e8a535, user_id=9cd14f6027a2471e8eb79d0e434f7e2b, role_id=b870cd1acaa744e1ae1d58b9ac283821)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:54.370783 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3a0587-ef4f-492a-886c-e973994f0bdc admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:54.505574 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 970/1929] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 08:56:54 2022] PUT /identity/v3/projects/27be6cbbc389495ca1f74ef2b5e8a535/users/9cd14f6027a2471e8eb79d0e434f7e2b/roles/b870cd1acaa744e1ae1d58b9ac283821 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:54.511131 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3a0587-ef4f-492a-886c-e973994f0bdc admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:54.574207 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:54.618965 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-42402f8d-02ad-485c-a22c-196a234ef913 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8238e09838d47d1bd59e0e7e7a0fab2'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:54.621510 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-42402f8d-02ad-485c-a22c-196a234ef913 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8238e09838d47d1bd59e0e7e7a0fab2'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:54.622441 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-42402f8d-02ad-485c-a22c-196a234ef913 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8238e09838d47d1bd59e0e7e7a0fab2'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:54.647266 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3a0587-ef4f-492a-886c-e973994f0bdc admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:54.659808 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-42402f8d-02ad-485c-a22c-196a234ef913 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8238e09838d47d1bd59e0e7e7a0fab2'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:54.673131 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb8a33c-89ea-4ed7-9736-844d8553fa3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:54.674540 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb8a33c-89ea-4ed7-9736-844d8553fa3a admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:54.674968 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=80,keystone:INSERT=3,keystone:DELETE=17 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:54.675091 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb8a33c-89ea-4ed7-9736-844d8553fa3a admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:54.675692 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb8a33c-89ea-4ed7-9736-844d8553fa3a admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:54.683434 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb8a33c-89ea-4ed7-9736-844d8553fa3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:54.686769 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb8a33c-89ea-4ed7-9736-844d8553fa3a admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:54.721250 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 971/1930] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:56:54 2022] GET /identity/v3/roles => generated 1420 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 08:56:54.727086 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb8a33c-89ea-4ed7-9736-844d8553fa3a admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:54.729526 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-cfb8a33c-89ea-4ed7-9736-844d8553fa3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:54.748376 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb8a33c-89ea-4ed7-9736-844d8553fa3a admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:54.773633 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-42402f8d-02ad-485c-a22c-196a234ef913 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8238e09838d47d1bd59e0e7e7a0fab2'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:54.791489 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766ed1df-5ceb-464d-a268-b951086ea98f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:54.793340 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766ed1df-5ceb-464d-a268-b951086ea98f admin admin] REQUEST_METHOD: `PUT` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:54.794168 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766ed1df-5ceb-464d-a268-b951086ea98f admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:54.798845 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766ed1df-5ceb-464d-a268-b951086ea98f admin admin] PATH_INFO: `/v3/projects/27be6cbbc389495ca1f74ef2b5e8a535/users/9cd14f6027a2471e8eb79d0e434f7e2b/roles/b870cd1acaa744e1ae1d58b9ac283821` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:54.799381 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766ed1df-5ceb-464d-a268-b951086ea98f admin admin] RBAC: Authorizing `identity:create_grant(project_id=27be6cbbc389495ca1f74ef2b5e8a535, user_id=9cd14f6027a2471e8eb79d0e434f7e2b, role_id=b870cd1acaa744e1ae1d58b9ac283821)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:54.806827 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766ed1df-5ceb-464d-a268-b951086ea98f admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:54.817088 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 960/1931] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 08:56:54 2022] DELETE /identity/v3/users/f8238e09838d47d1bd59e0e7e7a0fab2 => generated 0 bytes in 684 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:54.833339 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 972/1932] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 08:56:54 2022] PUT /identity/v3/projects/27be6cbbc389495ca1f74ef2b5e8a535/users/9cd14f6027a2471e8eb79d0e434f7e2b/roles/b870cd1acaa744e1ae1d58b9ac283821 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:54.860442 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbc2665-29a1-48a0-9d59-3aeed5fd6bd4 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:54.861354 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbc2665-29a1-48a0-9d59-3aeed5fd6bd4 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:54.861687 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbc2665-29a1-48a0-9d59-3aeed5fd6bd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:54.899830 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-9bbc2665-29a1-48a0-9d59-3aeed5fd6bd4 None None] MFA Rules not processed for user `9cd14f6027a2471e8eb79d0e434f7e2b`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:54.901355 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-9bbc2665-29a1-48a0-9d59-3aeed5fd6bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:55.026880 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 961/1933] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:56:54 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:56:55.040389 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766ed1df-5ceb-464d-a268-b951086ea98f admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:55.043176 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-766ed1df-5ceb-464d-a268-b951086ea98f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:55.130557 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766ed1df-5ceb-464d-a268-b951086ea98f admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:55.134980 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476eba31-4add-491e-bbb3-efdd52aa277f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbb9ac667f04c35a63b2689df18efed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:55.136071 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476eba31-4add-491e-bbb3-efdd52aa277f service cinder] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:55.136474 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476eba31-4add-491e-bbb3-efdd52aa277f service cinder] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:55.136905 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476eba31-4add-491e-bbb3-efdd52aa277f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:55.139174 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-476eba31-4add-491e-bbb3-efdd52aa277f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:55.307047 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476eba31-4add-491e-bbb3-efdd52aa277f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:55.323704 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476eba31-4add-491e-bbb3-efdd52aa277f service cinder] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:55.346850 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 973/1934] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:56:55 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:56:55.783478 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbc2665-29a1-48a0-9d59-3aeed5fd6bd4 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:55.787338 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-9bbc2665-29a1-48a0-9d59-3aeed5fd6bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:55.928724 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbc2665-29a1-48a0-9d59-3aeed5fd6bd4 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:55.935552 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71598a5-db96-45e4-8f30-f7d40c9d3a0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:55.936435 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71598a5-db96-45e4-8f30-f7d40c9d3a0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:55.936761 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71598a5-db96-45e4-8f30-f7d40c9d3a0a admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:55.937102 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71598a5-db96-45e4-8f30-f7d40c9d3a0a admin admin] PATH_INFO: `/v3/projects/204cfa7633604adabe6a2a3331405b53` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:55.938635 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71598a5-db96-45e4-8f30-f7d40c9d3a0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=204cfa7633604adabe6a2a3331405b53)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:55.940861 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71598a5-db96-45e4-8f30-f7d40c9d3a0a admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:56.104977 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-c71598a5-db96-45e4-8f30-f7d40c9d3a0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '204cfa7633604adabe6a2a3331405b53', 'request_id': 'req-c71598a5-db96-45e4-8f30-f7d40c9d3a0a'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:56.187888 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-c71598a5-db96-45e4-8f30-f7d40c9d3a0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '204cfa7633604adabe6a2a3331405b53', 'request_id': 'req-c71598a5-db96-45e4-8f30-f7d40c9d3a0a'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:56.210360 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-c71598a5-db96-45e4-8f30-f7d40c9d3a0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '204cfa7633604adabe6a2a3331405b53', 'request_id': 'req-c71598a5-db96-45e4-8f30-f7d40c9d3a0a'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:56.223001 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-c71598a5-db96-45e4-8f30-f7d40c9d3a0a admin admin] The token cache is being invalidate because project 204cfa7633604adabe6a2a3331405b53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:56:56.226858 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-c71598a5-db96-45e4-8f30-f7d40c9d3a0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:56.226858 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-c71598a5-db96-45e4-8f30-f7d40c9d3a0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:56.226858 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 962/1935] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 08:56:55 2022] DELETE /identity/v3/projects/204cfa7633604adabe6a2a3331405b53 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:56.231787 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48460d70-fba3-47b9-9b95-c4fb956b5936 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:56.232293 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48460d70-fba3-47b9-9b95-c4fb956b5936 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:56.233132 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48460d70-fba3-47b9-9b95-c4fb956b5936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:56.265815 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-48460d70-fba3-47b9-9b95-c4fb956b5936 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:56.267153 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-48460d70-fba3-47b9-9b95-c4fb956b5936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:56.347276 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 974/1936] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:56:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:56.356660 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d761ed-d78c-423f-89f3-17b27bc4795c None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:56.357142 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d761ed-d78c-423f-89f3-17b27bc4795c None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:56.357142 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d761ed-d78c-423f-89f3-17b27bc4795c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:56.393111 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:56.434262 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-62d761ed-d78c-423f-89f3-17b27bc4795c None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:56.439745 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-62d761ed-d78c-423f-89f3-17b27bc4795c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:56.456990 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 963/1937] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:56:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:56.463051 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48460d70-fba3-47b9-9b95-c4fb956b5936 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:56.475061 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:56.479634 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48460d70-fba3-47b9-9b95-c4fb956b5936 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:56.487017 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870987bb-95a3-4351-ab4f-75cfc24e1cb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:56.487990 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870987bb-95a3-4351-ab4f-75cfc24e1cb1 admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:56.489023 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870987bb-95a3-4351-ab4f-75cfc24e1cb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:56.489377 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870987bb-95a3-4351-ab4f-75cfc24e1cb1 admin admin] PATH_INFO: `/v3/domains` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:56.490118 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870987bb-95a3-4351-ab4f-75cfc24e1cb1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:56:56.490619 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870987bb-95a3-4351-ab4f-75cfc24e1cb1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:56.492568 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870987bb-95a3-4351-ab4f-75cfc24e1cb1 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:56.512898 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 975/1938] 10.222.0.55 () {58 vars in 1153 bytes} [Wed Aug 24 08:56:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:56:56.513930 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d761ed-d78c-423f-89f3-17b27bc4795c None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:56.524497 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d761ed-d78c-423f-89f3-17b27bc4795c None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:56.530535 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14add37-c583-449e-a9df-086f6c84bd88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:56.531207 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14add37-c583-449e-a9df-086f6c84bd88 admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:56.531503 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14add37-c583-449e-a9df-086f6c84bd88 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:56.531994 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14add37-c583-449e-a9df-086f6c84bd88 admin admin] PATH_INFO: `/v3/projects` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:56.532746 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14add37-c583-449e-a9df-086f6c84bd88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:56.534848 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14add37-c583-449e-a9df-086f6c84bd88 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:56.699113 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 964/1939] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 08:56:56 2022] POST /identity/v3/projects => generated 373 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:56:56.704675 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870987bb-95a3-4351-ab4f-75cfc24e1cb1 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:56.718353 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870987bb-95a3-4351-ab4f-75cfc24e1cb1 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:56.804846 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e111db-d1c6-43d5-a8d9-f64013ac1c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:56.805931 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e111db-d1c6-43d5-a8d9-f64013ac1c61 admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:56.806404 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e111db-d1c6-43d5-a8d9-f64013ac1c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:56.806835 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e111db-d1c6-43d5-a8d9-f64013ac1c61 admin admin] PATH_INFO: `/v3/users` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:56.807837 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e111db-d1c6-43d5-a8d9-f64013ac1c61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:56.810190 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e111db-d1c6-43d5-a8d9-f64013ac1c61 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:56.935365 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 976/1940] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 08:56:56 2022] POST /identity/v3/users => generated 287 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:56:56.942480 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14add37-c583-449e-a9df-086f6c84bd88 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:56.964780 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14add37-c583-449e-a9df-086f6c84bd88 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:56.981418 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85080ff2-ce95-47d5-94b0-1e41efb58a59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:56.982635 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85080ff2-ce95-47d5-94b0-1e41efb58a59 admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:56.982635 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85080ff2-ce95-47d5-94b0-1e41efb58a59 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:56.982635 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85080ff2-ce95-47d5-94b0-1e41efb58a59 admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:56.983232 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85080ff2-ce95-47d5-94b0-1e41efb58a59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:56.986210 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85080ff2-ce95-47d5-94b0-1e41efb58a59 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:57.007769 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 965/1941] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:56:56 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:56:57.013435 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e111db-d1c6-43d5-a8d9-f64013ac1c61 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:57.027283 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e111db-d1c6-43d5-a8d9-f64013ac1c61 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:57.044041 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bef2f26-acb8-4c9e-903c-88e029b8113a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:57.045569 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bef2f26-acb8-4c9e-903c-88e029b8113a admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:57.047211 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bef2f26-acb8-4c9e-903c-88e029b8113a admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:57.047554 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bef2f26-acb8-4c9e-903c-88e029b8113a admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:57.049199 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bef2f26-acb8-4c9e-903c-88e029b8113a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:57.055400 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bef2f26-acb8-4c9e-903c-88e029b8113a admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:57.084686 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 977/1942] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:56:57 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:56:57.085806 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85080ff2-ce95-47d5-94b0-1e41efb58a59 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:57.097055 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85080ff2-ce95-47d5-94b0-1e41efb58a59 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:57.115015 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daca8bbc-fda7-478f-8f2a-1c6f5ae79301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:57.116145 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daca8bbc-fda7-478f-8f2a-1c6f5ae79301 admin admin] REQUEST_METHOD: `PUT` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:57.116737 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daca8bbc-fda7-478f-8f2a-1c6f5ae79301 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:57.117279 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daca8bbc-fda7-478f-8f2a-1c6f5ae79301 admin admin] PATH_INFO: `/v3/projects/25791f0ae12d47a2afad44ccc262c35f/users/071c2b1a79d74af0bb4551d9ba00eeee/roles/b674fd0a592e4e0cada811bba5ceea4d` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:57.136932 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daca8bbc-fda7-478f-8f2a-1c6f5ae79301 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25791f0ae12d47a2afad44ccc262c35f, user_id=071c2b1a79d74af0bb4551d9ba00eeee, role_id=b674fd0a592e4e0cada811bba5ceea4d)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:57.141554 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daca8bbc-fda7-478f-8f2a-1c6f5ae79301 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:57.256898 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 966/1943] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 08:56:57 2022] PUT /identity/v3/projects/25791f0ae12d47a2afad44ccc262c35f/users/071c2b1a79d74af0bb4551d9ba00eeee/roles/b674fd0a592e4e0cada811bba5ceea4d => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:56:57.267147 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4956038-92e7-4279-a333-2b73dbee3d6c None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:57.267902 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4956038-92e7-4279-a333-2b73dbee3d6c None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:57.268318 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4956038-92e7-4279-a333-2b73dbee3d6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:57.310010 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-b4956038-92e7-4279-a333-2b73dbee3d6c None None] MFA Rules not processed for user `071c2b1a79d74af0bb4551d9ba00eeee`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:57.311429 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-b4956038-92e7-4279-a333-2b73dbee3d6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:57.413461 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 978/1944] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:56:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:57.431963 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daca8bbc-fda7-478f-8f2a-1c6f5ae79301 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:57.434653 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-daca8bbc-fda7-478f-8f2a-1c6f5ae79301 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:57.562571 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daca8bbc-fda7-478f-8f2a-1c6f5ae79301 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:57.570696 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d108bdeb-c463-4cd2-af63-186abc80596d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbb9ac667f04c35a63b2689df18efed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:57.579584 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d108bdeb-c463-4cd2-af63-186abc80596d service cinder] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:57.579584 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d108bdeb-c463-4cd2-af63-186abc80596d service cinder] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:57.579584 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d108bdeb-c463-4cd2-af63-186abc80596d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:57.589508 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d108bdeb-c463-4cd2-af63-186abc80596d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:57.592592 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d108bdeb-c463-4cd2-af63-186abc80596d service cinder] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:57.617622 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 967/1945] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:56:57 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:56:57.646436 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4956038-92e7-4279-a333-2b73dbee3d6c None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:57.767578 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4956038-92e7-4279-a333-2b73dbee3d6c None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:57.777595 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcaae11-028e-4d37-b04e-12f71bf9c10e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:57.778402 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcaae11-028e-4d37-b04e-12f71bf9c10e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:57.779051 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcaae11-028e-4d37-b04e-12f71bf9c10e admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:57.779396 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcaae11-028e-4d37-b04e-12f71bf9c10e admin admin] PATH_INFO: `/v3/users/071c2b1a79d74af0bb4551d9ba00eeee` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:57.780819 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcaae11-028e-4d37-b04e-12f71bf9c10e admin admin] RBAC: Authorizing `identity:delete_user(user_id=071c2b1a79d74af0bb4551d9ba00eeee)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:57.783021 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcaae11-028e-4d37-b04e-12f71bf9c10e admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:57.785628 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:58.111970 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-7bcaae11-028e-4d37-b04e-12f71bf9c10e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '071c2b1a79d74af0bb4551d9ba00eeee'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:58.113627 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-7bcaae11-028e-4d37-b04e-12f71bf9c10e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '071c2b1a79d74af0bb4551d9ba00eeee'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:58.115378 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-7bcaae11-028e-4d37-b04e-12f71bf9c10e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '071c2b1a79d74af0bb4551d9ba00eeee'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:58.169331 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-7bcaae11-028e-4d37-b04e-12f71bf9c10e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '071c2b1a79d74af0bb4551d9ba00eeee'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:58.273854 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-7bcaae11-028e-4d37-b04e-12f71bf9c10e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '071c2b1a79d74af0bb4551d9ba00eeee'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:58.305194 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 979/1946] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 08:56:57 2022] DELETE /identity/v3/users/071c2b1a79d74af0bb4551d9ba00eeee => generated 0 bytes in 659 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:56:58.310852 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3b4e8f-8bcb-422c-985a-17d3409de3b4 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:58.311649 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3b4e8f-8bcb-422c-985a-17d3409de3b4 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:58.311649 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3b4e8f-8bcb-422c-985a-17d3409de3b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:58.338261 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-aa3b4e8f-8bcb-422c-985a-17d3409de3b4 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:58.339651 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-aa3b4e8f-8bcb-422c-985a-17d3409de3b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:58.442824 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 968/1947] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:56:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:56:58.453957 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcaae11-028e-4d37-b04e-12f71bf9c10e admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:58.456261 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-7bcaae11-028e-4d37-b04e-12f71bf9c10e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:58.550584 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcaae11-028e-4d37-b04e-12f71bf9c10e admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:58.554777 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d42119-eb1e-4a29-922f-2b080f0a2b3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:58.555426 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d42119-eb1e-4a29-922f-2b080f0a2b3e service neutron] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:58.555693 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d42119-eb1e-4a29-922f-2b080f0a2b3e service neutron] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:58.556049 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d42119-eb1e-4a29-922f-2b080f0a2b3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:58.576733 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d42119-eb1e-4a29-922f-2b080f0a2b3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:58.578288 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d42119-eb1e-4a29-922f-2b080f0a2b3e service neutron] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:58.590537 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 980/1948] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:56:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:56:59.225252 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3b4e8f-8bcb-422c-985a-17d3409de3b4 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:59.227016 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-aa3b4e8f-8bcb-422c-985a-17d3409de3b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:59.247676 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3b4e8f-8bcb-422c-985a-17d3409de3b4 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:59.255703 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc3e005-5991-4073-9e63-fd46fc824d06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:59.256676 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc3e005-5991-4073-9e63-fd46fc824d06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:59.257145 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc3e005-5991-4073-9e63-fd46fc824d06 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:59.257746 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc3e005-5991-4073-9e63-fd46fc824d06 admin admin] PATH_INFO: `/v3/projects/25791f0ae12d47a2afad44ccc262c35f` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:59.259312 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc3e005-5991-4073-9e63-fd46fc824d06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25791f0ae12d47a2afad44ccc262c35f)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:59.263224 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc3e005-5991-4073-9e63-fd46fc824d06 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:59.418304 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:59.432290 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-ecc3e005-5991-4073-9e63-fd46fc824d06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25791f0ae12d47a2afad44ccc262c35f', 'request_id': 'req-ecc3e005-5991-4073-9e63-fd46fc824d06'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:59.536719 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-ecc3e005-5991-4073-9e63-fd46fc824d06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25791f0ae12d47a2afad44ccc262c35f', 'request_id': 'req-ecc3e005-5991-4073-9e63-fd46fc824d06'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:59.545172 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-ecc3e005-5991-4073-9e63-fd46fc824d06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25791f0ae12d47a2afad44ccc262c35f', 'request_id': 'req-ecc3e005-5991-4073-9e63-fd46fc824d06'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:59.558573 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-ecc3e005-5991-4073-9e63-fd46fc824d06 admin admin] The token cache is being invalidate because project 25791f0ae12d47a2afad44ccc262c35f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:56:59.560361 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-ecc3e005-5991-4073-9e63-fd46fc824d06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:59.561324 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-ecc3e005-5991-4073-9e63-fd46fc824d06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:59.563234 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 969/1949] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 08:56:59 2022] DELETE /identity/v3/projects/25791f0ae12d47a2afad44ccc262c35f => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:59.568629 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c735a50-e5d3-47ea-9b88-dd9f8f177364 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:59.569537 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c735a50-e5d3-47ea-9b88-dd9f8f177364 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:59.569747 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c735a50-e5d3-47ea-9b88-dd9f8f177364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:59.626481 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-0c735a50-e5d3-47ea-9b88-dd9f8f177364 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:59.627734 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-0c735a50-e5d3-47ea-9b88-dd9f8f177364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:59.775525 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 981/1950] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:56:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:59.784630 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46405662-3d5d-4af9-8a7d-a53ba8654cc6 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:59.784871 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46405662-3d5d-4af9-8a7d-a53ba8654cc6 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:59.785090 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46405662-3d5d-4af9-8a7d-a53ba8654cc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:59.807987 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-46405662-3d5d-4af9-8a7d-a53ba8654cc6 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:59.809344 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-46405662-3d5d-4af9-8a7d-a53ba8654cc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:59.818827 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 970/1951] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:56:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:59.823455 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c735a50-e5d3-47ea-9b88-dd9f8f177364 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:59.835752 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c735a50-e5d3-47ea-9b88-dd9f8f177364 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:59.840752 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17494caf-a22e-4c84-bd53-e10798df3636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:59.841548 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17494caf-a22e-4c84-bd53-e10798df3636 admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:59.841833 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17494caf-a22e-4c84-bd53-e10798df3636 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:59.842108 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17494caf-a22e-4c84-bd53-e10798df3636 admin admin] PATH_INFO: `/v3/domains` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:59.842795 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17494caf-a22e-4c84-bd53-e10798df3636 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:56:59.843103 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17494caf-a22e-4c84-bd53-e10798df3636 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:59.845381 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17494caf-a22e-4c84-bd53-e10798df3636 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:59.866894 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 982/1952] 10.222.0.55 () {58 vars in 1153 bytes} [Wed Aug 24 08:56:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:56:59.867718 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46405662-3d5d-4af9-8a7d-a53ba8654cc6 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:59.878261 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46405662-3d5d-4af9-8a7d-a53ba8654cc6 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:59.884774 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2978d2f5-fea7-4f7b-87e2-1c013c5e6b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:59.886754 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2978d2f5-fea7-4f7b-87e2-1c013c5e6b24 admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:59.886754 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2978d2f5-fea7-4f7b-87e2-1c013c5e6b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:59.886754 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2978d2f5-fea7-4f7b-87e2-1c013c5e6b24 admin admin] PATH_INFO: `/v3/projects` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:59.886754 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2978d2f5-fea7-4f7b-87e2-1c013c5e6b24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:59.888650 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2978d2f5-fea7-4f7b-87e2-1c013c5e6b24 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:00.005985 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 971/1953] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 08:56:59 2022] POST /identity/v3/projects => generated 383 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:57:00.007455 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17494caf-a22e-4c84-bd53-e10798df3636 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:00.019072 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17494caf-a22e-4c84-bd53-e10798df3636 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:00.025090 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:00.082219 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8213a02-5661-447d-8ba0-656b84c4096b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:00.082740 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8213a02-5661-447d-8ba0-656b84c4096b admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:00.083018 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8213a02-5661-447d-8ba0-656b84c4096b admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:00.083290 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8213a02-5661-447d-8ba0-656b84c4096b admin admin] PATH_INFO: `/v3/users` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:00.083919 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8213a02-5661-447d-8ba0-656b84c4096b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:00.085869 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8213a02-5661-447d-8ba0-656b84c4096b admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:00.177506 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 983/1954] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 08:57:00 2022] POST /identity/v3/users => generated 292 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:57:00.182016 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2978d2f5-fea7-4f7b-87e2-1c013c5e6b24 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:00.195170 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2978d2f5-fea7-4f7b-87e2-1c013c5e6b24 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:00.201082 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4c659b-3263-4d84-be94-69aee5543485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:00.202008 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4c659b-3263-4d84-be94-69aee5543485 admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:00.202338 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4c659b-3263-4d84-be94-69aee5543485 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:00.202617 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4c659b-3263-4d84-be94-69aee5543485 admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:00.203321 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4c659b-3263-4d84-be94-69aee5543485 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:00.205317 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4c659b-3263-4d84-be94-69aee5543485 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:00.220841 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 972/1955] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:57:00 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 08:57:00.222141 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8213a02-5661-447d-8ba0-656b84c4096b admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:00.239870 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8213a02-5661-447d-8ba0-656b84c4096b admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:00.245438 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28895ce7-1abe-4019-b5ad-30ea42f5f72a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:00.246118 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28895ce7-1abe-4019-b5ad-30ea42f5f72a admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:00.246403 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28895ce7-1abe-4019-b5ad-30ea42f5f72a admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:00.246681 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28895ce7-1abe-4019-b5ad-30ea42f5f72a admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:00.247345 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28895ce7-1abe-4019-b5ad-30ea42f5f72a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:00.249107 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28895ce7-1abe-4019-b5ad-30ea42f5f72a admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:00.262852 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 984/1956] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:57:00 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:57:00.270312 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4c659b-3263-4d84-be94-69aee5543485 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:00.283796 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4c659b-3263-4d84-be94-69aee5543485 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:00.290492 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14bbcc4-abc5-479b-b29c-0bd65b2010d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:00.291778 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14bbcc4-abc5-479b-b29c-0bd65b2010d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:00.292244 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14bbcc4-abc5-479b-b29c-0bd65b2010d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:00.292716 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14bbcc4-abc5-479b-b29c-0bd65b2010d2 admin admin] PATH_INFO: `/v3/projects/c1977f516dfe40f7b26ffae80d68edd9/users/c11f1ac1772b46d2a03165571edb6f10/roles/b674fd0a592e4e0cada811bba5ceea4d` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:00.325486 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14bbcc4-abc5-479b-b29c-0bd65b2010d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1977f516dfe40f7b26ffae80d68edd9, user_id=c11f1ac1772b46d2a03165571edb6f10, role_id=b674fd0a592e4e0cada811bba5ceea4d)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:00.331823 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14bbcc4-abc5-479b-b29c-0bd65b2010d2 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:00.683493 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 973/1957] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 08:57:00 2022] PUT /identity/v3/projects/c1977f516dfe40f7b26ffae80d68edd9/users/c11f1ac1772b46d2a03165571edb6f10/roles/b674fd0a592e4e0cada811bba5ceea4d => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:57:00.688829 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c622f9b-feee-4fc8-bd3d-2e497684a62b None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:00.689327 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c622f9b-feee-4fc8-bd3d-2e497684a62b None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:00.689436 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c622f9b-feee-4fc8-bd3d-2e497684a62b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:00.713586 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-4c622f9b-feee-4fc8-bd3d-2e497684a62b None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:00.714877 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-4c622f9b-feee-4fc8-bd3d-2e497684a62b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:00.816185 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 985/1958] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:57:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:57:00.826452 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14bbcc4-abc5-479b-b29c-0bd65b2010d2 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:00.828341 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-f14bbcc4-abc5-479b-b29c-0bd65b2010d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:00.890616 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14bbcc4-abc5-479b-b29c-0bd65b2010d2 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:00.898967 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82ced92-bedb-4eda-864f-256873da152e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:00.899859 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82ced92-bedb-4eda-864f-256873da152e service neutron] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:00.900160 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82ced92-bedb-4eda-864f-256873da152e service neutron] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:00.905131 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82ced92-bedb-4eda-864f-256873da152e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:00.919008 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82ced92-bedb-4eda-864f-256873da152e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:00.920382 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82ced92-bedb-4eda-864f-256873da152e service neutron] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:00.933044 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 974/1959] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:57:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:57:02.721796 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ca81ab-b47c-4c7d-8c6a-7aa9d5f61783 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:02.722372 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ca81ab-b47c-4c7d-8c6a-7aa9d5f61783 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:02.722372 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ca81ab-b47c-4c7d-8c6a-7aa9d5f61783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:02.757119 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-45ca81ab-b47c-4c7d-8c6a-7aa9d5f61783 None None] MFA Rules not processed for user `c11f1ac1772b46d2a03165571edb6f10`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:02.765981 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-45ca81ab-b47c-4c7d-8c6a-7aa9d5f61783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:02.895816 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 986/1960] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:57:02 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:57:02.906850 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82ced92-bedb-4eda-864f-256873da152e service neutron] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:02.908470 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-e82ced92-bedb-4eda-864f-256873da152e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:02.972132 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82ced92-bedb-4eda-864f-256873da152e service neutron] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:02.976700 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9172c97-1ec8-4335-afa9-6fbad095bd10 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbb9ac667f04c35a63b2689df18efed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:02.977339 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9172c97-1ec8-4335-afa9-6fbad095bd10 service cinder] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:02.977658 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9172c97-1ec8-4335-afa9-6fbad095bd10 service cinder] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:02.977986 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9172c97-1ec8-4335-afa9-6fbad095bd10 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:02.988601 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9172c97-1ec8-4335-afa9-6fbad095bd10 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:02.990088 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9172c97-1ec8-4335-afa9-6fbad095bd10 service cinder] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:03.001559 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 975/1961] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:57:02 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:57:04.700622 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ca81ab-b47c-4c7d-8c6a-7aa9d5f61783 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:04.702932 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-45ca81ab-b47c-4c7d-8c6a-7aa9d5f61783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:04.729827 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ca81ab-b47c-4c7d-8c6a-7aa9d5f61783 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:04.735233 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8ab685-f1b5-40d1-b985-e315cc8020e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbb9ac667f04c35a63b2689df18efed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:04.736214 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8ab685-f1b5-40d1-b985-e315cc8020e6 service cinder] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:04.736751 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8ab685-f1b5-40d1-b985-e315cc8020e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:04.737283 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8ab685-f1b5-40d1-b985-e315cc8020e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:04.741911 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-1e8ab685-f1b5-40d1-b985-e315cc8020e6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:04.764665 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:04.845960 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8ab685-f1b5-40d1-b985-e315cc8020e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:04.847763 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8ab685-f1b5-40d1-b985-e315cc8020e6 service cinder] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:04.859851 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 987/1962] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:57:04 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:57:06.079633 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e5decc-e3ad-4e3e-b10a-7d656c0595b4 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:06.080184 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e5decc-e3ad-4e3e-b10a-7d656c0595b4 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:06.080184 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e5decc-e3ad-4e3e-b10a-7d656c0595b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:06.095623 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:06.104345 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-52e5decc-e3ad-4e3e-b10a-7d656c0595b4 None None] MFA Rules not processed for user `c11f1ac1772b46d2a03165571edb6f10`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:06.105717 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-52e5decc-e3ad-4e3e-b10a-7d656c0595b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:06.116328 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 976/1963] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:57:06 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:57:13.591220 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8ab685-f1b5-40d1-b985-e315cc8020e6 service cinder] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:13.593836 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-1e8ab685-f1b5-40d1-b985-e315cc8020e6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:13.625511 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8ab685-f1b5-40d1-b985-e315cc8020e6 service cinder] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:13.630380 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e90366-a4fa-49ed-9c66-f1f451434294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:13.631053 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e90366-a4fa-49ed-9c66-f1f451434294 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:13.631839 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e90366-a4fa-49ed-9c66-f1f451434294 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:13.631839 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e90366-a4fa-49ed-9c66-f1f451434294 admin admin] PATH_INFO: `/v3/users/66863e52e6d94b159814fa4426394628` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:13.633159 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e90366-a4fa-49ed-9c66-f1f451434294 admin admin] RBAC: Authorizing `identity:delete_user(user_id=66863e52e6d94b159814fa4426394628)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:13.635138 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e90366-a4fa-49ed-9c66-f1f451434294 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:13.946580 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-35e90366-a4fa-49ed-9c66-f1f451434294 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66863e52e6d94b159814fa4426394628'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:13.948169 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-35e90366-a4fa-49ed-9c66-f1f451434294 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66863e52e6d94b159814fa4426394628'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:13.950004 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-35e90366-a4fa-49ed-9c66-f1f451434294 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66863e52e6d94b159814fa4426394628'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:13.981123 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-35e90366-a4fa-49ed-9c66-f1f451434294 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66863e52e6d94b159814fa4426394628'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:14.106522 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-35e90366-a4fa-49ed-9c66-f1f451434294 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66863e52e6d94b159814fa4426394628'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:14.118938 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 988/1964] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 08:57:13 2022] DELETE /identity/v3/users/66863e52e6d94b159814fa4426394628 => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:57:14.124794 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e848fd6-7b66-4cce-8067-ad28a7eb759f None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:14.125550 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e848fd6-7b66-4cce-8067-ad28a7eb759f None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:14.125550 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e848fd6-7b66-4cce-8067-ad28a7eb759f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:14.145843 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-4e848fd6-7b66-4cce-8067-ad28a7eb759f None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:14.147066 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-4e848fd6-7b66-4cce-8067-ad28a7eb759f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:14.224017 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 977/1965] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:57:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:57:14.234648 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e90366-a4fa-49ed-9c66-f1f451434294 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:14.237142 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-35e90366-a4fa-49ed-9c66-f1f451434294 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:14.329117 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e90366-a4fa-49ed-9c66-f1f451434294 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:14.333405 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f31a190-c719-4467-9e16-db76a06fb372 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:14.334265 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f31a190-c719-4467-9e16-db76a06fb372 service neutron] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:14.334585 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f31a190-c719-4467-9e16-db76a06fb372 service neutron] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:14.334986 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f31a190-c719-4467-9e16-db76a06fb372 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:14.346338 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f31a190-c719-4467-9e16-db76a06fb372 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:14.347859 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f31a190-c719-4467-9e16-db76a06fb372 service neutron] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:14.357966 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 989/1966] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:57:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:57:14.900732 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e848fd6-7b66-4cce-8067-ad28a7eb759f None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:14.900732 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-4e848fd6-7b66-4cce-8067-ad28a7eb759f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:14.917454 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e848fd6-7b66-4cce-8067-ad28a7eb759f None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:14.929128 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fc8942-69dc-4f17-8640-0e14fa2bab61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:14.930566 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fc8942-69dc-4f17-8640-0e14fa2bab61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:14.931205 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fc8942-69dc-4f17-8640-0e14fa2bab61 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:14.931900 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fc8942-69dc-4f17-8640-0e14fa2bab61 admin admin] PATH_INFO: `/v3/projects/b503eeea4d854d30a5107e6d65048fb8` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:14.934984 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fc8942-69dc-4f17-8640-0e14fa2bab61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b503eeea4d854d30a5107e6d65048fb8)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:14.942727 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fc8942-69dc-4f17-8640-0e14fa2bab61 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:15.255826 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-60fc8942-69dc-4f17-8640-0e14fa2bab61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b503eeea4d854d30a5107e6d65048fb8', 'request_id': 'req-60fc8942-69dc-4f17-8640-0e14fa2bab61'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:15.336941 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-60fc8942-69dc-4f17-8640-0e14fa2bab61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b503eeea4d854d30a5107e6d65048fb8', 'request_id': 'req-60fc8942-69dc-4f17-8640-0e14fa2bab61'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:15.349806 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-60fc8942-69dc-4f17-8640-0e14fa2bab61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b503eeea4d854d30a5107e6d65048fb8', 'request_id': 'req-60fc8942-69dc-4f17-8640-0e14fa2bab61'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:15.365647 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-60fc8942-69dc-4f17-8640-0e14fa2bab61 admin admin] The token cache is being invalidate because project b503eeea4d854d30a5107e6d65048fb8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:57:15.366095 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-60fc8942-69dc-4f17-8640-0e14fa2bab61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:15.366858 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-60fc8942-69dc-4f17-8640-0e14fa2bab61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:15.368973 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 978/1967] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 08:57:14 2022] DELETE /identity/v3/projects/b503eeea4d854d30a5107e6d65048fb8 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:57:15.374632 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f31a190-c719-4467-9e16-db76a06fb372 service neutron] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:15.391644 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f31a190-c719-4467-9e16-db76a06fb372 service neutron] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:15.454817 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:15.498081 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1a3269-0cdf-42a2-a58e-a6928f2d1c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:15.498936 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1a3269-0cdf-42a2-a58e-a6928f2d1c82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:15.499348 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1a3269-0cdf-42a2-a58e-a6928f2d1c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:15.499791 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1a3269-0cdf-42a2-a58e-a6928f2d1c82 admin admin] PATH_INFO: `/v3/users/9cd14f6027a2471e8eb79d0e434f7e2b` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:15.501256 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1a3269-0cdf-42a2-a58e-a6928f2d1c82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9cd14f6027a2471e8eb79d0e434f7e2b)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:15.504633 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1a3269-0cdf-42a2-a58e-a6928f2d1c82 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:15.829229 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-2c1a3269-0cdf-42a2-a58e-a6928f2d1c82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9cd14f6027a2471e8eb79d0e434f7e2b'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:15.830205 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-2c1a3269-0cdf-42a2-a58e-a6928f2d1c82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9cd14f6027a2471e8eb79d0e434f7e2b'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:15.831014 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-2c1a3269-0cdf-42a2-a58e-a6928f2d1c82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9cd14f6027a2471e8eb79d0e434f7e2b'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:15.855599 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-2c1a3269-0cdf-42a2-a58e-a6928f2d1c82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9cd14f6027a2471e8eb79d0e434f7e2b'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:15.978720 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-2c1a3269-0cdf-42a2-a58e-a6928f2d1c82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9cd14f6027a2471e8eb79d0e434f7e2b'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:15.995311 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 990/1968] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 08:57:15 2022] DELETE /identity/v3/users/9cd14f6027a2471e8eb79d0e434f7e2b => generated 0 bytes in 620 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:57:16.596937 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fc8942-69dc-4f17-8640-0e14fa2bab61 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:16.598461 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-60fc8942-69dc-4f17-8640-0e14fa2bab61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:16.613193 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fc8942-69dc-4f17-8640-0e14fa2bab61 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:16.686343 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942622b7-29ef-4f9f-a707-6d04410c1f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:16.687019 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942622b7-29ef-4f9f-a707-6d04410c1f3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:16.687285 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942622b7-29ef-4f9f-a707-6d04410c1f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:16.687581 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942622b7-29ef-4f9f-a707-6d04410c1f3b admin admin] PATH_INFO: `/v3/projects/27be6cbbc389495ca1f74ef2b5e8a535` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:16.688859 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942622b7-29ef-4f9f-a707-6d04410c1f3b admin admin] RBAC: Authorizing `identity:delete_project(project_id=27be6cbbc389495ca1f74ef2b5e8a535)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:16.690849 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942622b7-29ef-4f9f-a707-6d04410c1f3b admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:16.789329 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:16.810316 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-942622b7-29ef-4f9f-a707-6d04410c1f3b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27be6cbbc389495ca1f74ef2b5e8a535', 'request_id': 'req-942622b7-29ef-4f9f-a707-6d04410c1f3b'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:16.905275 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-942622b7-29ef-4f9f-a707-6d04410c1f3b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27be6cbbc389495ca1f74ef2b5e8a535', 'request_id': 'req-942622b7-29ef-4f9f-a707-6d04410c1f3b'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:16.912991 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-942622b7-29ef-4f9f-a707-6d04410c1f3b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27be6cbbc389495ca1f74ef2b5e8a535', 'request_id': 'req-942622b7-29ef-4f9f-a707-6d04410c1f3b'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:16.925850 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-942622b7-29ef-4f9f-a707-6d04410c1f3b admin admin] The token cache is being invalidate because project 27be6cbbc389495ca1f74ef2b5e8a535 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:57:16.925850 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-942622b7-29ef-4f9f-a707-6d04410c1f3b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:16.926482 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-942622b7-29ef-4f9f-a707-6d04410c1f3b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:16.928124 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 979/1969] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 08:57:16 2022] DELETE /identity/v3/projects/27be6cbbc389495ca1f74ef2b5e8a535 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:57:16.934410 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48717a59-bd75-4c41-9bc8-7b54826cb993 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:16.934973 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48717a59-bd75-4c41-9bc8-7b54826cb993 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:16.935104 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48717a59-bd75-4c41-9bc8-7b54826cb993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:16.957936 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-48717a59-bd75-4c41-9bc8-7b54826cb993 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:16.959219 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-48717a59-bd75-4c41-9bc8-7b54826cb993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:17.096492 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 991/1970] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:57:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:57:17.100860 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fbba76-6da7-4b3a-8155-ff6b5017aae9 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:17.101148 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fbba76-6da7-4b3a-8155-ff6b5017aae9 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:17.101426 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fbba76-6da7-4b3a-8155-ff6b5017aae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:17.122889 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-25fbba76-6da7-4b3a-8155-ff6b5017aae9 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:17.124114 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-25fbba76-6da7-4b3a-8155-ff6b5017aae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:17.132277 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 980/1971] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:57:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:57:17.136779 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48717a59-bd75-4c41-9bc8-7b54826cb993 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:17.147689 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48717a59-bd75-4c41-9bc8-7b54826cb993 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:17.152818 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a52dd0-066f-4cd5-a4db-b33b0d00da0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:17.153383 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a52dd0-066f-4cd5-a4db-b33b0d00da0f admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:17.153665 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a52dd0-066f-4cd5-a4db-b33b0d00da0f admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:17.154191 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a52dd0-066f-4cd5-a4db-b33b0d00da0f admin admin] PATH_INFO: `/v3/domains` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:17.154861 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a52dd0-066f-4cd5-a4db-b33b0d00da0f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:57:17.155461 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a52dd0-066f-4cd5-a4db-b33b0d00da0f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:17.157151 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a52dd0-066f-4cd5-a4db-b33b0d00da0f admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:17.176207 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 992/1972] 10.222.0.55 () {58 vars in 1153 bytes} [Wed Aug 24 08:57:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:57:17.181353 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fbba76-6da7-4b3a-8155-ff6b5017aae9 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:17.197970 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fbba76-6da7-4b3a-8155-ff6b5017aae9 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:17.207535 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bb8f1d-d7b7-4725-a231-248a8e0f1eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:17.208768 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bb8f1d-d7b7-4725-a231-248a8e0f1eaf admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:17.209249 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bb8f1d-d7b7-4725-a231-248a8e0f1eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:17.209708 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bb8f1d-d7b7-4725-a231-248a8e0f1eaf admin admin] PATH_INFO: `/v3/projects` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:17.211047 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bb8f1d-d7b7-4725-a231-248a8e0f1eaf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:17.214405 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bb8f1d-d7b7-4725-a231-248a8e0f1eaf admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:17.325427 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 981/1973] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 08:57:17 2022] POST /identity/v3/projects => generated 375 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:57:17.329798 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a52dd0-066f-4cd5-a4db-b33b0d00da0f admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:17.341251 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a52dd0-066f-4cd5-a4db-b33b0d00da0f admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:17.391624 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:17.431521 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c909290a-f806-4f89-80ee-616fd66685dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:17.432350 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c909290a-f806-4f89-80ee-616fd66685dc admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:17.432741 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c909290a-f806-4f89-80ee-616fd66685dc admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:17.433110 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c909290a-f806-4f89-80ee-616fd66685dc admin admin] PATH_INFO: `/v3/users` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:17.433845 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c909290a-f806-4f89-80ee-616fd66685dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:17.436899 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c909290a-f806-4f89-80ee-616fd66685dc admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:17.596150 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 993/1974] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 08:57:17 2022] POST /identity/v3/users => generated 288 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:57:17.597593 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bb8f1d-d7b7-4725-a231-248a8e0f1eaf admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:17.617236 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bb8f1d-d7b7-4725-a231-248a8e0f1eaf admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:17.625059 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a29a45-f858-4687-ab0d-c897bebd9f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:17.626555 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a29a45-f858-4687-ab0d-c897bebd9f76 admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:17.627049 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a29a45-f858-4687-ab0d-c897bebd9f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:17.627539 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a29a45-f858-4687-ab0d-c897bebd9f76 admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:17.629316 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a29a45-f858-4687-ab0d-c897bebd9f76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:17.632551 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a29a45-f858-4687-ab0d-c897bebd9f76 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:17.653033 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 982/1975] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:57:17 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:57:17.662870 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c909290a-f806-4f89-80ee-616fd66685dc admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:17.681071 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c909290a-f806-4f89-80ee-616fd66685dc admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:17.688856 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae46cae-5506-4c87-98ed-a9d297fdf4c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:17.700152 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae46cae-5506-4c87-98ed-a9d297fdf4c3 admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:17.700647 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae46cae-5506-4c87-98ed-a9d297fdf4c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:17.700939 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae46cae-5506-4c87-98ed-a9d297fdf4c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:17.701760 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae46cae-5506-4c87-98ed-a9d297fdf4c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:17.703568 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae46cae-5506-4c87-98ed-a9d297fdf4c3 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:17.716330 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 994/1976] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:57:17 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 08:57:17.721126 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a29a45-f858-4687-ab0d-c897bebd9f76 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:17.744588 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a29a45-f858-4687-ab0d-c897bebd9f76 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:17.754291 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612246ef-252a-4fe0-96ae-727f26f6100e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:17.755818 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612246ef-252a-4fe0-96ae-727f26f6100e admin admin] REQUEST_METHOD: `PUT` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:17.756310 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612246ef-252a-4fe0-96ae-727f26f6100e admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:17.756894 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612246ef-252a-4fe0-96ae-727f26f6100e admin admin] PATH_INFO: `/v3/projects/81db7fdc34dc4964a383c1b22e757934/users/315fd1ff671a48e2945289e247782ae1/roles/b674fd0a592e4e0cada811bba5ceea4d` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:17.786916 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612246ef-252a-4fe0-96ae-727f26f6100e admin admin] RBAC: Authorizing `identity:create_grant(project_id=81db7fdc34dc4964a383c1b22e757934, user_id=315fd1ff671a48e2945289e247782ae1, role_id=b674fd0a592e4e0cada811bba5ceea4d)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:17.790399 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612246ef-252a-4fe0-96ae-727f26f6100e admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:17.975752 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 983/1977] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 08:57:17 2022] PUT /identity/v3/projects/81db7fdc34dc4964a383c1b22e757934/users/315fd1ff671a48e2945289e247782ae1/roles/b674fd0a592e4e0cada811bba5ceea4d => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:57:17.987545 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f26438-8c4f-499f-adf5-3dff6ad97397 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:17.989240 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f26438-8c4f-499f-adf5-3dff6ad97397 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:17.989240 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f26438-8c4f-499f-adf5-3dff6ad97397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:18.020504 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-35f26438-8c4f-499f-adf5-3dff6ad97397 None None] MFA Rules not processed for user `315fd1ff671a48e2945289e247782ae1`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:18.022033 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-35f26438-8c4f-499f-adf5-3dff6ad97397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:18.104670 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 995/1978] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:57:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:57:18.119106 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612246ef-252a-4fe0-96ae-727f26f6100e admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:18.121141 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-612246ef-252a-4fe0-96ae-727f26f6100e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:18.225524 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612246ef-252a-4fe0-96ae-727f26f6100e admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:18.230292 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3748415c-844f-47ce-8693-18b8b314327c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5512043ca9a7421b821741ca88d50b07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:18.230917 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3748415c-844f-47ce-8693-18b8b314327c service glance] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:18.231238 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3748415c-844f-47ce-8693-18b8b314327c service glance] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:18.231516 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3748415c-844f-47ce-8693-18b8b314327c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:18.242960 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3748415c-844f-47ce-8693-18b8b314327c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:18.245038 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3748415c-844f-47ce-8693-18b8b314327c service glance] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:18.267176 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 984/1979] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:57:18 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:57:22.969965 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f26438-8c4f-499f-adf5-3dff6ad97397 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:22.971521 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-35f26438-8c4f-499f-adf5-3dff6ad97397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:23.074001 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f26438-8c4f-499f-adf5-3dff6ad97397 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:23.079113 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c94ff14-24ab-4cba-b950-7024d44152ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:23.079780 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c94ff14-24ab-4cba-b950-7024d44152ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:23.080393 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c94ff14-24ab-4cba-b950-7024d44152ab admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:23.080393 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c94ff14-24ab-4cba-b950-7024d44152ab admin admin] PATH_INFO: `/v3/users/718a0dfdef714e358a74ad8e424b1b8a` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:23.082326 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c94ff14-24ab-4cba-b950-7024d44152ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=718a0dfdef714e358a74ad8e424b1b8a)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:23.084401 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c94ff14-24ab-4cba-b950-7024d44152ab admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:23.277170 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:23.280388 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-1c94ff14-24ab-4cba-b950-7024d44152ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '718a0dfdef714e358a74ad8e424b1b8a'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:23.281108 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-1c94ff14-24ab-4cba-b950-7024d44152ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '718a0dfdef714e358a74ad8e424b1b8a'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:23.281710 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-1c94ff14-24ab-4cba-b950-7024d44152ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '718a0dfdef714e358a74ad8e424b1b8a'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:23.295561 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-1c94ff14-24ab-4cba-b950-7024d44152ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '718a0dfdef714e358a74ad8e424b1b8a'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:23.441440 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-1c94ff14-24ab-4cba-b950-7024d44152ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '718a0dfdef714e358a74ad8e424b1b8a'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:23.462154 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 996/1980] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 08:57:22 2022] DELETE /identity/v3/users/718a0dfdef714e358a74ad8e424b1b8a => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:57:23.911906 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3748415c-844f-47ce-8693-18b8b314327c service glance] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:23.945869 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3748415c-844f-47ce-8693-18b8b314327c service glance] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:24.007583 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:24.044239 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d7a58c-b429-40a1-8ab2-f793987d7712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:24.044944 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d7a58c-b429-40a1-8ab2-f793987d7712 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:24.045245 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d7a58c-b429-40a1-8ab2-f793987d7712 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:24.045528 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d7a58c-b429-40a1-8ab2-f793987d7712 admin admin] PATH_INFO: `/v3/projects/d0c61eee777c46779632c08ee11946c5` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:24.047455 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d7a58c-b429-40a1-8ab2-f793987d7712 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0c61eee777c46779632c08ee11946c5)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:24.048953 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d7a58c-b429-40a1-8ab2-f793987d7712 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:24.221367 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-59d7a58c-b429-40a1-8ab2-f793987d7712 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0c61eee777c46779632c08ee11946c5', 'request_id': 'req-59d7a58c-b429-40a1-8ab2-f793987d7712'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:24.333345 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-59d7a58c-b429-40a1-8ab2-f793987d7712 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0c61eee777c46779632c08ee11946c5', 'request_id': 'req-59d7a58c-b429-40a1-8ab2-f793987d7712'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:24.342843 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-59d7a58c-b429-40a1-8ab2-f793987d7712 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0c61eee777c46779632c08ee11946c5', 'request_id': 'req-59d7a58c-b429-40a1-8ab2-f793987d7712'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:24.358325 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-59d7a58c-b429-40a1-8ab2-f793987d7712 admin admin] The token cache is being invalidate because project d0c61eee777c46779632c08ee11946c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:57:24.358902 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-59d7a58c-b429-40a1-8ab2-f793987d7712 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:24.360118 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-59d7a58c-b429-40a1-8ab2-f793987d7712 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:24.361059 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 985/1981] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 08:57:23 2022] DELETE /identity/v3/projects/d0c61eee777c46779632c08ee11946c5 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:57:24.366821 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba84460-8fe6-4edc-9a0b-8e42168e4273 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:24.367332 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba84460-8fe6-4edc-9a0b-8e42168e4273 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:24.367421 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba84460-8fe6-4edc-9a0b-8e42168e4273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:24.393354 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-8ba84460-8fe6-4edc-9a0b-8e42168e4273 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:24.394754 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-8ba84460-8fe6-4edc-9a0b-8e42168e4273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:24.466564 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 997/1982] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:57:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:57:24.475508 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8582b73-7e23-4a82-8b6a-9733b05adb05 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:24.475804 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8582b73-7e23-4a82-8b6a-9733b05adb05 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:24.476030 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8582b73-7e23-4a82-8b6a-9733b05adb05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:24.502275 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-b8582b73-7e23-4a82-8b6a-9733b05adb05 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:24.503586 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-b8582b73-7e23-4a82-8b6a-9733b05adb05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:24.511881 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 986/1983] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:57:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:57:24.516586 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba84460-8fe6-4edc-9a0b-8e42168e4273 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:24.529329 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba84460-8fe6-4edc-9a0b-8e42168e4273 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:24.536879 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cccab94-e276-453a-b2c6-1ec689b52595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:24.537513 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cccab94-e276-453a-b2c6-1ec689b52595 admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:24.537794 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cccab94-e276-453a-b2c6-1ec689b52595 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:24.538065 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cccab94-e276-453a-b2c6-1ec689b52595 admin admin] PATH_INFO: `/v3/domains` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:24.538732 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cccab94-e276-453a-b2c6-1ec689b52595 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:57:24.539035 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cccab94-e276-453a-b2c6-1ec689b52595 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:24.540778 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cccab94-e276-453a-b2c6-1ec689b52595 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:24.554431 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 998/1984] 10.222.0.55 () {58 vars in 1153 bytes} [Wed Aug 24 08:57:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:57:24.559390 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8582b73-7e23-4a82-8b6a-9733b05adb05 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:24.569558 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8582b73-7e23-4a82-8b6a-9733b05adb05 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:24.574755 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7624c8d1-5b5a-4357-902e-1ec9b610b9f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:24.575409 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7624c8d1-5b5a-4357-902e-1ec9b610b9f8 admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:24.575688 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7624c8d1-5b5a-4357-902e-1ec9b610b9f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:24.575964 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7624c8d1-5b5a-4357-902e-1ec9b610b9f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:24.576687 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7624c8d1-5b5a-4357-902e-1ec9b610b9f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:24.578600 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7624c8d1-5b5a-4357-902e-1ec9b610b9f8 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:24.698781 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 987/1985] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 08:57:24 2022] POST /identity/v3/projects => generated 371 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:57:24.703636 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cccab94-e276-453a-b2c6-1ec689b52595 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:24.715724 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cccab94-e276-453a-b2c6-1ec689b52595 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:24.783482 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99a7402-fb40-4693-8ca0-a546a51722bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:24.784202 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99a7402-fb40-4693-8ca0-a546a51722bf admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:24.785257 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99a7402-fb40-4693-8ca0-a546a51722bf admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:24.795429 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99a7402-fb40-4693-8ca0-a546a51722bf admin admin] PATH_INFO: `/v3/users` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:24.795429 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99a7402-fb40-4693-8ca0-a546a51722bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:24.795429 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99a7402-fb40-4693-8ca0-a546a51722bf admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:24.972578 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 999/1986] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 08:57:24 2022] POST /identity/v3/users => generated 286 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:57:24.976446 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7624c8d1-5b5a-4357-902e-1ec9b610b9f8 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:24.987613 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7624c8d1-5b5a-4357-902e-1ec9b610b9f8 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:24.992620 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0619acec-78a8-4016-99d3-06adff1c4a1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:24.993256 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0619acec-78a8-4016-99d3-06adff1c4a1d admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:24.993536 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0619acec-78a8-4016-99d3-06adff1c4a1d admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:24.993812 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0619acec-78a8-4016-99d3-06adff1c4a1d admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:24.994472 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0619acec-78a8-4016-99d3-06adff1c4a1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:24.996107 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0619acec-78a8-4016-99d3-06adff1c4a1d admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:25.013739 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 988/1987] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:57:24 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:57:25.018225 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99a7402-fb40-4693-8ca0-a546a51722bf admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:25.030271 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99a7402-fb40-4693-8ca0-a546a51722bf admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:25.036311 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dbdeba-8396-4c6d-a96f-ff9615d43ed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:25.037025 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dbdeba-8396-4c6d-a96f-ff9615d43ed3 admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:25.037376 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dbdeba-8396-4c6d-a96f-ff9615d43ed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:25.037824 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dbdeba-8396-4c6d-a96f-ff9615d43ed3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:25.038776 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dbdeba-8396-4c6d-a96f-ff9615d43ed3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:25.040990 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dbdeba-8396-4c6d-a96f-ff9615d43ed3 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:25.053928 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1000/1988] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:57:25 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 08:57:25.058677 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0619acec-78a8-4016-99d3-06adff1c4a1d admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:25.070448 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0619acec-78a8-4016-99d3-06adff1c4a1d admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:25.075624 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1931c878-8c09-4eb4-a925-b991dcc6f50b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:25.076316 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1931c878-8c09-4eb4-a925-b991dcc6f50b admin admin] REQUEST_METHOD: `PUT` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:25.076668 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1931c878-8c09-4eb4-a925-b991dcc6f50b admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:25.076945 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1931c878-8c09-4eb4-a925-b991dcc6f50b admin admin] PATH_INFO: `/v3/projects/af77427688444a6389044f863be8130d/users/e79d556cfe1e4260a34ad8fadb08e27e/roles/b674fd0a592e4e0cada811bba5ceea4d` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:25.098517 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1931c878-8c09-4eb4-a925-b991dcc6f50b admin admin] RBAC: Authorizing `identity:create_grant(project_id=af77427688444a6389044f863be8130d, user_id=e79d556cfe1e4260a34ad8fadb08e27e, role_id=b674fd0a592e4e0cada811bba5ceea4d)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:25.101309 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1931c878-8c09-4eb4-a925-b991dcc6f50b admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:25.182850 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 989/1989] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 08:57:25 2022] PUT /identity/v3/projects/af77427688444a6389044f863be8130d/users/e79d556cfe1e4260a34ad8fadb08e27e/roles/b674fd0a592e4e0cada811bba5ceea4d => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:57:25.201692 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f17b5ee-fd48-4fa1-a059-ffc4c762a390 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:25.202172 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f17b5ee-fd48-4fa1-a059-ffc4c762a390 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:25.202656 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f17b5ee-fd48-4fa1-a059-ffc4c762a390 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:25.248171 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-8f17b5ee-fd48-4fa1-a059-ffc4c762a390 None None] MFA Rules not processed for user `e79d556cfe1e4260a34ad8fadb08e27e`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:25.249619 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-8f17b5ee-fd48-4fa1-a059-ffc4c762a390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:25.265175 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:25.331663 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1001/1990] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:57:25 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:57:25.344105 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1931c878-8c09-4eb4-a925-b991dcc6f50b admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:25.346224 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-1931c878-8c09-4eb4-a925-b991dcc6f50b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:25.422414 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:25.441054 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1931c878-8c09-4eb4-a925-b991dcc6f50b admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:25.449717 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d987ce83-0491-400c-99bc-711bb4b23576 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17da357e555420799ee681bffbcc268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:25.450346 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d987ce83-0491-400c-99bc-711bb4b23576 service nova] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:25.453296 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d987ce83-0491-400c-99bc-711bb4b23576 service nova] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:25.453296 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d987ce83-0491-400c-99bc-711bb4b23576 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:25.459952 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d987ce83-0491-400c-99bc-711bb4b23576 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:25.461898 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d987ce83-0491-400c-99bc-711bb4b23576 service nova] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:25.473816 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 990/1991] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:57:25 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:57:30.082544 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f17b5ee-fd48-4fa1-a059-ffc4c762a390 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:30.177235 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f17b5ee-fd48-4fa1-a059-ffc4c762a390 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:30.184339 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1c73e3-b4c7-483d-832d-37206d354f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:30.185555 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1c73e3-b4c7-483d-832d-37206d354f6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:30.186025 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1c73e3-b4c7-483d-832d-37206d354f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:30.186607 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1c73e3-b4c7-483d-832d-37206d354f6d admin admin] PATH_INFO: `/v3/users/e79d556cfe1e4260a34ad8fadb08e27e` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:30.188239 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1c73e3-b4c7-483d-832d-37206d354f6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e79d556cfe1e4260a34ad8fadb08e27e)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:30.191246 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1c73e3-b4c7-483d-832d-37206d354f6d admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:30.277964 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a738fd80-2143-4d8d-b01f-407a0266ec74 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:30.278619 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a738fd80-2143-4d8d-b01f-407a0266ec74 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:30.278619 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a738fd80-2143-4d8d-b01f-407a0266ec74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:30.303313 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-a738fd80-2143-4d8d-b01f-407a0266ec74 None None] MFA Rules not processed for user `bb815fa9f45a4b1db1f30fbfaeb3c4c9`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:30.304811 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-a738fd80-2143-4d8d-b01f-407a0266ec74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:30.365162 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 991/1992] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:57:30 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:57:30.472817 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-7f1c73e3-b4c7-483d-832d-37206d354f6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e79d556cfe1e4260a34ad8fadb08e27e'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:30.473468 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-7f1c73e3-b4c7-483d-832d-37206d354f6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e79d556cfe1e4260a34ad8fadb08e27e'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:30.474093 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-7f1c73e3-b4c7-483d-832d-37206d354f6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e79d556cfe1e4260a34ad8fadb08e27e'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:30.490364 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-7f1c73e3-b4c7-483d-832d-37206d354f6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e79d556cfe1e4260a34ad8fadb08e27e'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:30.631536 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-7f1c73e3-b4c7-483d-832d-37206d354f6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e79d556cfe1e4260a34ad8fadb08e27e'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:30.650331 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1002/1993] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 08:57:30 2022] DELETE /identity/v3/users/e79d556cfe1e4260a34ad8fadb08e27e => generated 0 bytes in 571 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:57:30.656286 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85973af5-551c-41a3-b9e9-f0fe8dc815ab None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:30.658811 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85973af5-551c-41a3-b9e9-f0fe8dc815ab None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:30.658811 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85973af5-551c-41a3-b9e9-f0fe8dc815ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:30.688968 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-85973af5-551c-41a3-b9e9-f0fe8dc815ab None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:30.690231 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-85973af5-551c-41a3-b9e9-f0fe8dc815ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:30.795049 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 992/1994] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:57:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:57:30.805791 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1c73e3-b4c7-483d-832d-37206d354f6d admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:30.807994 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-7f1c73e3-b4c7-483d-832d-37206d354f6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:30.857985 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:30.888478 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1c73e3-b4c7-483d-832d-37206d354f6d admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:30.893654 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14542b6a-2383-4c83-8cd6-9b30c6987933 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:30.894381 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14542b6a-2383-4c83-8cd6-9b30c6987933 service neutron] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:30.894904 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14542b6a-2383-4c83-8cd6-9b30c6987933 service neutron] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:30.895260 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14542b6a-2383-4c83-8cd6-9b30c6987933 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:30.911044 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14542b6a-2383-4c83-8cd6-9b30c6987933 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:30.912799 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14542b6a-2383-4c83-8cd6-9b30c6987933 service neutron] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:30.924663 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1003/1995] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:57:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:57:31.495036 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85973af5-551c-41a3-b9e9-f0fe8dc815ab None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:31.497384 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-85973af5-551c-41a3-b9e9-f0fe8dc815ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:31.515049 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85973af5-551c-41a3-b9e9-f0fe8dc815ab None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:31.521878 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57db3248-38c7-4700-8d40-af153e45bd08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:31.522612 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57db3248-38c7-4700-8d40-af153e45bd08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:31.525271 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57db3248-38c7-4700-8d40-af153e45bd08 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:31.525271 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57db3248-38c7-4700-8d40-af153e45bd08 admin admin] PATH_INFO: `/v3/projects/af77427688444a6389044f863be8130d` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:31.525271 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57db3248-38c7-4700-8d40-af153e45bd08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af77427688444a6389044f863be8130d)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:31.528076 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57db3248-38c7-4700-8d40-af153e45bd08 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:31.679109 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-57db3248-38c7-4700-8d40-af153e45bd08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af77427688444a6389044f863be8130d', 'request_id': 'req-57db3248-38c7-4700-8d40-af153e45bd08'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:31.748062 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-57db3248-38c7-4700-8d40-af153e45bd08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af77427688444a6389044f863be8130d', 'request_id': 'req-57db3248-38c7-4700-8d40-af153e45bd08'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:31.756565 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-57db3248-38c7-4700-8d40-af153e45bd08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af77427688444a6389044f863be8130d', 'request_id': 'req-57db3248-38c7-4700-8d40-af153e45bd08'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:31.770867 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-57db3248-38c7-4700-8d40-af153e45bd08 admin admin] The token cache is being invalidate because project af77427688444a6389044f863be8130d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:57:31.771376 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-57db3248-38c7-4700-8d40-af153e45bd08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:31.772247 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-57db3248-38c7-4700-8d40-af153e45bd08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:31.773864 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 993/1996] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 08:57:31 2022] DELETE /identity/v3/projects/af77427688444a6389044f863be8130d => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:57:31.778948 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87719efe-a9d4-42f2-a2b8-da0dcc2c5ff9 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:31.779793 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87719efe-a9d4-42f2-a2b8-da0dcc2c5ff9 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:31.779793 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87719efe-a9d4-42f2-a2b8-da0dcc2c5ff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:31.802253 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-87719efe-a9d4-42f2-a2b8-da0dcc2c5ff9 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:31.803574 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-87719efe-a9d4-42f2-a2b8-da0dcc2c5ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:31.888276 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1004/1997] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:57:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:57:31.902876 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830bca89-b875-49ae-bc5b-7487d3e0a6da None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:31.903195 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830bca89-b875-49ae-bc5b-7487d3e0a6da None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:31.903475 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830bca89-b875-49ae-bc5b-7487d3e0a6da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:31.932438 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-830bca89-b875-49ae-bc5b-7487d3e0a6da None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:31.934772 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-830bca89-b875-49ae-bc5b-7487d3e0a6da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:31.955000 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 994/1998] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:57:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:57:31.958990 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87719efe-a9d4-42f2-a2b8-da0dcc2c5ff9 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:31.976928 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87719efe-a9d4-42f2-a2b8-da0dcc2c5ff9 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:31.982234 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e27ba7c-eda8-49f2-bc06-e26b2df60f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:31.983121 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e27ba7c-eda8-49f2-bc06-e26b2df60f71 admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:31.983402 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e27ba7c-eda8-49f2-bc06-e26b2df60f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:31.983674 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e27ba7c-eda8-49f2-bc06-e26b2df60f71 admin admin] PATH_INFO: `/v3/domains` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:31.984358 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e27ba7c-eda8-49f2-bc06-e26b2df60f71 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:57:31.984764 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e27ba7c-eda8-49f2-bc06-e26b2df60f71 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:31.986541 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e27ba7c-eda8-49f2-bc06-e26b2df60f71 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:32.004988 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1005/1999] 10.222.0.55 () {58 vars in 1153 bytes} [Wed Aug 24 08:57:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:57:32.009910 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830bca89-b875-49ae-bc5b-7487d3e0a6da None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:32.021486 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830bca89-b875-49ae-bc5b-7487d3e0a6da None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:32.026466 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e6f2a5-a443-4376-8c4b-a7c407c89a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:32.027204 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e6f2a5-a443-4376-8c4b-a7c407c89a2b admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:32.027483 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e6f2a5-a443-4376-8c4b-a7c407c89a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:32.027822 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e6f2a5-a443-4376-8c4b-a7c407c89a2b admin admin] PATH_INFO: `/v3/projects` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:32.028387 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e6f2a5-a443-4376-8c4b-a7c407c89a2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:32.030604 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e6f2a5-a443-4376-8c4b-a7c407c89a2b admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:32.035932 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:32.178372 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 995/2000] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 08:57:32 2022] POST /identity/v3/projects => generated 365 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:57:32.183213 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e27ba7c-eda8-49f2-bc06-e26b2df60f71 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:32.194663 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e27ba7c-eda8-49f2-bc06-e26b2df60f71 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:32.265268 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90f4ea5-06b8-416e-84bc-1d671ccb40da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:32.265899 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90f4ea5-06b8-416e-84bc-1d671ccb40da admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:32.266173 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90f4ea5-06b8-416e-84bc-1d671ccb40da admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:32.266441 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90f4ea5-06b8-416e-84bc-1d671ccb40da admin admin] PATH_INFO: `/v3/users` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:32.267068 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90f4ea5-06b8-416e-84bc-1d671ccb40da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:32.269074 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90f4ea5-06b8-416e-84bc-1d671ccb40da admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:32.394179 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1006/2001] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 08:57:32 2022] POST /identity/v3/users => generated 283 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:57:32.400109 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e6f2a5-a443-4376-8c4b-a7c407c89a2b admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:32.419750 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e6f2a5-a443-4376-8c4b-a7c407c89a2b admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:32.428310 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1af7cd6-9645-4d3f-ae15-6be36f17ae33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:32.429427 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af7cd6-9645-4d3f-ae15-6be36f17ae33 admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:32.429940 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af7cd6-9645-4d3f-ae15-6be36f17ae33 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:32.430296 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af7cd6-9645-4d3f-ae15-6be36f17ae33 admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:32.431381 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1af7cd6-9645-4d3f-ae15-6be36f17ae33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:32.434470 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1af7cd6-9645-4d3f-ae15-6be36f17ae33 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:32.454385 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 996/2002] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:57:32 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:57:32.458776 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90f4ea5-06b8-416e-84bc-1d671ccb40da admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:32.475107 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90f4ea5-06b8-416e-84bc-1d671ccb40da admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:32.483500 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3900eb39-b9e2-4b04-8786-d11d2637ab0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:32.484471 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3900eb39-b9e2-4b04-8786-d11d2637ab0e admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:32.484944 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3900eb39-b9e2-4b04-8786-d11d2637ab0e admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:32.485375 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3900eb39-b9e2-4b04-8786-d11d2637ab0e admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:32.486549 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3900eb39-b9e2-4b04-8786-d11d2637ab0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:32.490255 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3900eb39-b9e2-4b04-8786-d11d2637ab0e admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:32.511291 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1007/2003] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:57:32 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 08:57:32.516839 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1af7cd6-9645-4d3f-ae15-6be36f17ae33 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:32.536914 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1af7cd6-9645-4d3f-ae15-6be36f17ae33 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:32.546017 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee2bb8c-e214-43df-9c1c-b655ecaa235b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:32.547131 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee2bb8c-e214-43df-9c1c-b655ecaa235b admin admin] REQUEST_METHOD: `PUT` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:32.547611 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee2bb8c-e214-43df-9c1c-b655ecaa235b admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:32.548076 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee2bb8c-e214-43df-9c1c-b655ecaa235b admin admin] PATH_INFO: `/v3/projects/ac57026c44b64930b9dc98edb821512e/users/895fff9e244d4060b97b8e1065cd066f/roles/b674fd0a592e4e0cada811bba5ceea4d` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:32.577057 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee2bb8c-e214-43df-9c1c-b655ecaa235b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac57026c44b64930b9dc98edb821512e, user_id=895fff9e244d4060b97b8e1065cd066f, role_id=b674fd0a592e4e0cada811bba5ceea4d)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:32.582335 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee2bb8c-e214-43df-9c1c-b655ecaa235b admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:32.672165 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 997/2004] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 08:57:32 2022] PUT /identity/v3/projects/ac57026c44b64930b9dc98edb821512e/users/895fff9e244d4060b97b8e1065cd066f/roles/b674fd0a592e4e0cada811bba5ceea4d => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:57:32.686383 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b213fcdb-2306-464f-a582-cbc203fefa01 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:32.686572 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b213fcdb-2306-464f-a582-cbc203fefa01 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:32.686842 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b213fcdb-2306-464f-a582-cbc203fefa01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:32.715536 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-b213fcdb-2306-464f-a582-cbc203fefa01 None None] MFA Rules not processed for user `895fff9e244d4060b97b8e1065cd066f`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:32.717094 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-b213fcdb-2306-464f-a582-cbc203fefa01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:32.764944 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:32.806939 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1008/2005] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:57:32 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:57:32.812222 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee2bb8c-e214-43df-9c1c-b655ecaa235b admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:32.885698 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee2bb8c-e214-43df-9c1c-b655ecaa235b admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:32.891331 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2cc7f1-b9c5-42b5-92dd-9519682c3721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:32.892146 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2cc7f1-b9c5-42b5-92dd-9519682c3721 admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:32.892435 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2cc7f1-b9c5-42b5-92dd-9519682c3721 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:32.892761 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2cc7f1-b9c5-42b5-92dd-9519682c3721 admin admin] PATH_INFO: `/v3/projects` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:32.894025 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2cc7f1-b9c5-42b5-92dd-9519682c3721 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:32.896566 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2cc7f1-b9c5-42b5-92dd-9519682c3721 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:33.013358 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 998/2006] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 08:57:32 2022] POST /identity/v3/projects => generated 363 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:57:33.022900 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b213fcdb-2306-464f-a582-cbc203fefa01 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:33.035899 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b213fcdb-2306-464f-a582-cbc203fefa01 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:33.119643 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc393046-51e1-49ec-ae64-1e5d2e48e0ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:33.119643 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc393046-51e1-49ec-ae64-1e5d2e48e0ed admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:33.119643 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc393046-51e1-49ec-ae64-1e5d2e48e0ed admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:33.119643 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc393046-51e1-49ec-ae64-1e5d2e48e0ed admin admin] PATH_INFO: `/v3/users` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:33.120208 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc393046-51e1-49ec-ae64-1e5d2e48e0ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:33.123201 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc393046-51e1-49ec-ae64-1e5d2e48e0ed admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:33.200390 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1009/2007] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 08:57:33 2022] POST /identity/v3/users => generated 288 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:57:33.203995 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2cc7f1-b9c5-42b5-92dd-9519682c3721 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:33.215490 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2cc7f1-b9c5-42b5-92dd-9519682c3721 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:33.221376 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3fb31f-2f82-47f1-b198-e1e23f1f5a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:33.221997 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3fb31f-2f82-47f1-b198-e1e23f1f5a3a admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:33.222280 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3fb31f-2f82-47f1-b198-e1e23f1f5a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:33.222575 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3fb31f-2f82-47f1-b198-e1e23f1f5a3a admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:33.223256 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3fb31f-2f82-47f1-b198-e1e23f1f5a3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:33.224981 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3fb31f-2f82-47f1-b198-e1e23f1f5a3a admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:33.236497 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 999/2008] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:57:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:57:33.241012 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc393046-51e1-49ec-ae64-1e5d2e48e0ed admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:33.251271 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc393046-51e1-49ec-ae64-1e5d2e48e0ed admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:33.256826 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c412df10-bb5f-4750-b3b9-d2e71cbae093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:33.257547 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c412df10-bb5f-4750-b3b9-d2e71cbae093 admin admin] REQUEST_METHOD: `PUT` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:33.257988 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c412df10-bb5f-4750-b3b9-d2e71cbae093 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:33.258318 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c412df10-bb5f-4750-b3b9-d2e71cbae093 admin admin] PATH_INFO: `/v3/projects/de3ff37e786845369e839f6ef25df588/users/1603f3d586bd4546989bda7c43d80e28/roles/b870cd1acaa744e1ae1d58b9ac283821` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:33.292719 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c412df10-bb5f-4750-b3b9-d2e71cbae093 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de3ff37e786845369e839f6ef25df588, user_id=1603f3d586bd4546989bda7c43d80e28, role_id=b870cd1acaa744e1ae1d58b9ac283821)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:33.297967 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c412df10-bb5f-4750-b3b9-d2e71cbae093 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:33.381810 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1010/2009] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 08:57:33 2022] PUT /identity/v3/projects/de3ff37e786845369e839f6ef25df588/users/1603f3d586bd4546989bda7c43d80e28/roles/b870cd1acaa744e1ae1d58b9ac283821 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:57:33.383839 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3fb31f-2f82-47f1-b198-e1e23f1f5a3a admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:33.403248 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3fb31f-2f82-47f1-b198-e1e23f1f5a3a admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:33.493145 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e087aa9a-0352-49ea-961d-8cf41253dc37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:33.493741 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e087aa9a-0352-49ea-961d-8cf41253dc37 admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:33.494026 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e087aa9a-0352-49ea-961d-8cf41253dc37 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:33.494299 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e087aa9a-0352-49ea-961d-8cf41253dc37 admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:33.494986 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e087aa9a-0352-49ea-961d-8cf41253dc37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:33.496646 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e087aa9a-0352-49ea-961d-8cf41253dc37 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:33.513277 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1000/2010] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:57:33 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 08:57:33.519363 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c412df10-bb5f-4750-b3b9-d2e71cbae093 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:33.538590 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c412df10-bb5f-4750-b3b9-d2e71cbae093 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:33.545622 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9892f76b-188d-4c95-b844-1c290a698a1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:33.546682 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9892f76b-188d-4c95-b844-1c290a698a1e admin admin] REQUEST_METHOD: `PUT` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:33.547104 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9892f76b-188d-4c95-b844-1c290a698a1e admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:33.547528 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9892f76b-188d-4c95-b844-1c290a698a1e admin admin] PATH_INFO: `/v3/projects/de3ff37e786845369e839f6ef25df588/users/1603f3d586bd4546989bda7c43d80e28/roles/b870cd1acaa744e1ae1d58b9ac283821` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:33.551370 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9892f76b-188d-4c95-b844-1c290a698a1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=de3ff37e786845369e839f6ef25df588, user_id=1603f3d586bd4546989bda7c43d80e28, role_id=b870cd1acaa744e1ae1d58b9ac283821)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:33.555744 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9892f76b-188d-4c95-b844-1c290a698a1e admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:33.571231 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1011/2011] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 08:57:33 2022] PUT /identity/v3/projects/de3ff37e786845369e839f6ef25df588/users/1603f3d586bd4546989bda7c43d80e28/roles/b870cd1acaa744e1ae1d58b9ac283821 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:57:33.582383 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f985f7-84a7-45f6-82e0-f6e3964e996d None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:33.582949 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f985f7-84a7-45f6-82e0-f6e3964e996d None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:33.582949 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f985f7-84a7-45f6-82e0-f6e3964e996d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:33.618010 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-b8f985f7-84a7-45f6-82e0-f6e3964e996d None None] MFA Rules not processed for user `1603f3d586bd4546989bda7c43d80e28`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:33.619690 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-b8f985f7-84a7-45f6-82e0-f6e3964e996d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:33.630886 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:33.714633 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1001/2012] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:57:33 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:57:33.727048 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9892f76b-188d-4c95-b844-1c290a698a1e admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:33.728849 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-9892f76b-188d-4c95-b844-1c290a698a1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:33.809070 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9892f76b-188d-4c95-b844-1c290a698a1e admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:33.813338 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4976fd70-ed0d-41bd-a74a-2a10b74dfcfd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbb9ac667f04c35a63b2689df18efed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:33.813970 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4976fd70-ed0d-41bd-a74a-2a10b74dfcfd service cinder] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:33.814253 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4976fd70-ed0d-41bd-a74a-2a10b74dfcfd service cinder] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:33.814528 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4976fd70-ed0d-41bd-a74a-2a10b74dfcfd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:33.825880 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4976fd70-ed0d-41bd-a74a-2a10b74dfcfd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:33.827736 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4976fd70-ed0d-41bd-a74a-2a10b74dfcfd service cinder] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:33.839371 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1012/2013] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:57:33 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:57:35.156478 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f985f7-84a7-45f6-82e0-f6e3964e996d None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:35.270519 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f985f7-84a7-45f6-82e0-f6e3964e996d None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:35.279036 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9528ca05-6ab6-4422-a4a2-ca6f7fc6d0f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:35.280530 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9528ca05-6ab6-4422-a4a2-ca6f7fc6d0f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:35.281189 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9528ca05-6ab6-4422-a4a2-ca6f7fc6d0f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:35.281660 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9528ca05-6ab6-4422-a4a2-ca6f7fc6d0f6 admin admin] PATH_INFO: `/v3/users/895fff9e244d4060b97b8e1065cd066f` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:35.285385 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9528ca05-6ab6-4422-a4a2-ca6f7fc6d0f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=895fff9e244d4060b97b8e1065cd066f)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:35.288453 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9528ca05-6ab6-4422-a4a2-ca6f7fc6d0f6 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:35.536429 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-9528ca05-6ab6-4422-a4a2-ca6f7fc6d0f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '895fff9e244d4060b97b8e1065cd066f'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:35.537593 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-9528ca05-6ab6-4422-a4a2-ca6f7fc6d0f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '895fff9e244d4060b97b8e1065cd066f'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:35.538625 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-9528ca05-6ab6-4422-a4a2-ca6f7fc6d0f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '895fff9e244d4060b97b8e1065cd066f'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:35.562627 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-9528ca05-6ab6-4422-a4a2-ca6f7fc6d0f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '895fff9e244d4060b97b8e1065cd066f'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:35.642204 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-9528ca05-6ab6-4422-a4a2-ca6f7fc6d0f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '895fff9e244d4060b97b8e1065cd066f'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:35.659779 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1002/2014] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 08:57:35 2022] DELETE /identity/v3/users/895fff9e244d4060b97b8e1065cd066f => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:57:35.664782 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4126e65-5e28-451a-8cbf-377e15e61bc4 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:35.665668 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4126e65-5e28-451a-8cbf-377e15e61bc4 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:35.665668 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4126e65-5e28-451a-8cbf-377e15e61bc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:35.692089 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-f4126e65-5e28-451a-8cbf-377e15e61bc4 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:35.693348 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-f4126e65-5e28-451a-8cbf-377e15e61bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:35.726631 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:35.770067 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1013/2015] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:57:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:57:35.781701 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9528ca05-6ab6-4422-a4a2-ca6f7fc6d0f6 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:35.784860 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-9528ca05-6ab6-4422-a4a2-ca6f7fc6d0f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:35.838542 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:35.873084 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9528ca05-6ab6-4422-a4a2-ca6f7fc6d0f6 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:35.877756 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432ac3c6-56e2-41e0-8f03-0382269a902a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:35.878756 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432ac3c6-56e2-41e0-8f03-0382269a902a service neutron] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:35.879038 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432ac3c6-56e2-41e0-8f03-0382269a902a service neutron] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:35.879313 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432ac3c6-56e2-41e0-8f03-0382269a902a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:35.895612 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432ac3c6-56e2-41e0-8f03-0382269a902a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:35.898382 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432ac3c6-56e2-41e0-8f03-0382269a902a service neutron] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:35.921292 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1003/2016] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:57:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:57:36.612659 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4126e65-5e28-451a-8cbf-377e15e61bc4 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:36.614634 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-f4126e65-5e28-451a-8cbf-377e15e61bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:36.633659 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4126e65-5e28-451a-8cbf-377e15e61bc4 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:36.644060 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd848c6-fed0-4a97-800f-76a4c3d1ebd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:36.645339 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd848c6-fed0-4a97-800f-76a4c3d1ebd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:36.648990 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd848c6-fed0-4a97-800f-76a4c3d1ebd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:36.648990 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd848c6-fed0-4a97-800f-76a4c3d1ebd2 admin admin] PATH_INFO: `/v3/projects/ac57026c44b64930b9dc98edb821512e` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:36.650554 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd848c6-fed0-4a97-800f-76a4c3d1ebd2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac57026c44b64930b9dc98edb821512e)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:36.653849 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd848c6-fed0-4a97-800f-76a4c3d1ebd2 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:36.818031 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-2dd848c6-fed0-4a97-800f-76a4c3d1ebd2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac57026c44b64930b9dc98edb821512e', 'request_id': 'req-2dd848c6-fed0-4a97-800f-76a4c3d1ebd2'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:36.827588 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-2dd848c6-fed0-4a97-800f-76a4c3d1ebd2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac57026c44b64930b9dc98edb821512e', 'request_id': 'req-2dd848c6-fed0-4a97-800f-76a4c3d1ebd2'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:36.912374 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-2dd848c6-fed0-4a97-800f-76a4c3d1ebd2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac57026c44b64930b9dc98edb821512e', 'request_id': 'req-2dd848c6-fed0-4a97-800f-76a4c3d1ebd2'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:36.928242 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-2dd848c6-fed0-4a97-800f-76a4c3d1ebd2 admin admin] The token cache is being invalidate because project ac57026c44b64930b9dc98edb821512e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:57:36.928717 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-2dd848c6-fed0-4a97-800f-76a4c3d1ebd2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:36.929658 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-2dd848c6-fed0-4a97-800f-76a4c3d1ebd2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:36.931424 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1014/2017] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 08:57:36 2022] DELETE /identity/v3/projects/ac57026c44b64930b9dc98edb821512e => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:57:36.937404 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432ac3c6-56e2-41e0-8f03-0382269a902a service neutron] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:36.952738 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432ac3c6-56e2-41e0-8f03-0382269a902a service neutron] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:37.023898 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b827a4b9-7141-4b14-9122-d98335f803ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:37.024566 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b827a4b9-7141-4b14-9122-d98335f803ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:37.024859 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b827a4b9-7141-4b14-9122-d98335f803ac admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:37.025133 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b827a4b9-7141-4b14-9122-d98335f803ac admin admin] PATH_INFO: `/v3/users/1603f3d586bd4546989bda7c43d80e28` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:37.026532 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b827a4b9-7141-4b14-9122-d98335f803ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=1603f3d586bd4546989bda7c43d80e28)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:37.028485 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b827a4b9-7141-4b14-9122-d98335f803ac admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:37.273124 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-b827a4b9-7141-4b14-9122-d98335f803ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1603f3d586bd4546989bda7c43d80e28'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:37.273791 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-b827a4b9-7141-4b14-9122-d98335f803ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1603f3d586bd4546989bda7c43d80e28'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:37.274404 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-b827a4b9-7141-4b14-9122-d98335f803ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1603f3d586bd4546989bda7c43d80e28'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:37.288508 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-b827a4b9-7141-4b14-9122-d98335f803ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1603f3d586bd4546989bda7c43d80e28'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:37.394392 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-b827a4b9-7141-4b14-9122-d98335f803ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1603f3d586bd4546989bda7c43d80e28'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:37.414004 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1004/2018] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 08:57:36 2022] DELETE /identity/v3/users/1603f3d586bd4546989bda7c43d80e28 => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:57:38.001157 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd848c6-fed0-4a97-800f-76a4c3d1ebd2 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:38.003238 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-2dd848c6-fed0-4a97-800f-76a4c3d1ebd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:38.020870 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd848c6-fed0-4a97-800f-76a4c3d1ebd2 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:38.108469 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77e0e60-1ad5-417f-b6ca-b3e8fa9c22ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:38.109178 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e0e60-1ad5-417f-b6ca-b3e8fa9c22ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:38.109459 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e0e60-1ad5-417f-b6ca-b3e8fa9c22ab admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:38.109732 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e0e60-1ad5-417f-b6ca-b3e8fa9c22ab admin admin] PATH_INFO: `/v3/projects/de3ff37e786845369e839f6ef25df588` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:38.111008 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77e0e60-1ad5-417f-b6ca-b3e8fa9c22ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=de3ff37e786845369e839f6ef25df588)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:38.113087 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77e0e60-1ad5-417f-b6ca-b3e8fa9c22ab admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:38.263356 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-d77e0e60-1ad5-417f-b6ca-b3e8fa9c22ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de3ff37e786845369e839f6ef25df588', 'request_id': 'req-d77e0e60-1ad5-417f-b6ca-b3e8fa9c22ab'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:38.269264 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:38.272327 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-d77e0e60-1ad5-417f-b6ca-b3e8fa9c22ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de3ff37e786845369e839f6ef25df588', 'request_id': 'req-d77e0e60-1ad5-417f-b6ca-b3e8fa9c22ab'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:38.360262 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-d77e0e60-1ad5-417f-b6ca-b3e8fa9c22ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de3ff37e786845369e839f6ef25df588', 'request_id': 'req-d77e0e60-1ad5-417f-b6ca-b3e8fa9c22ab'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:38.381984 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-d77e0e60-1ad5-417f-b6ca-b3e8fa9c22ab admin admin] The token cache is being invalidate because project de3ff37e786845369e839f6ef25df588 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:57:38.382541 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-d77e0e60-1ad5-417f-b6ca-b3e8fa9c22ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:38.383735 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-d77e0e60-1ad5-417f-b6ca-b3e8fa9c22ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:38.385566 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1015/2019] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 08:57:37 2022] DELETE /identity/v3/projects/de3ff37e786845369e839f6ef25df588 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:57:38.390788 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5456d8fa-0e91-44ff-ba74-9d9ce4cfff73 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:38.391612 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5456d8fa-0e91-44ff-ba74-9d9ce4cfff73 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:38.391772 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5456d8fa-0e91-44ff-ba74-9d9ce4cfff73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:38.429223 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-5456d8fa-0e91-44ff-ba74-9d9ce4cfff73 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:38.431240 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-5456d8fa-0e91-44ff-ba74-9d9ce4cfff73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:38.500642 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:38.523872 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1005/2020] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:57:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:57:38.539225 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15dabe3-b21e-4b86-9c57-965bdebf82a5 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:38.539564 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15dabe3-b21e-4b86-9c57-965bdebf82a5 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:38.539889 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15dabe3-b21e-4b86-9c57-965bdebf82a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:38.572161 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-a15dabe3-b21e-4b86-9c57-965bdebf82a5 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:38.573458 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-a15dabe3-b21e-4b86-9c57-965bdebf82a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:38.582615 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1016/2021] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:57:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:57:38.587566 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5456d8fa-0e91-44ff-ba74-9d9ce4cfff73 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:38.599629 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5456d8fa-0e91-44ff-ba74-9d9ce4cfff73 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:38.604725 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39715e3-6253-4085-9d58-b8084302ddca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:38.605358 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39715e3-6253-4085-9d58-b8084302ddca admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:38.605785 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39715e3-6253-4085-9d58-b8084302ddca admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:38.606073 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39715e3-6253-4085-9d58-b8084302ddca admin admin] PATH_INFO: `/v3/domains` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:38.606751 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39715e3-6253-4085-9d58-b8084302ddca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:57:38.607051 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39715e3-6253-4085-9d58-b8084302ddca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:38.608750 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39715e3-6253-4085-9d58-b8084302ddca admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:38.624297 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1006/2022] 10.222.0.55 () {58 vars in 1153 bytes} [Wed Aug 24 08:57:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:57:38.629922 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15dabe3-b21e-4b86-9c57-965bdebf82a5 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:38.642832 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15dabe3-b21e-4b86-9c57-965bdebf82a5 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:38.648062 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be716b37-9e6d-4ca5-955d-def565889a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:38.648766 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be716b37-9e6d-4ca5-955d-def565889a13 admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:38.649028 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be716b37-9e6d-4ca5-955d-def565889a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:38.649304 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be716b37-9e6d-4ca5-955d-def565889a13 admin admin] PATH_INFO: `/v3/projects` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:38.649952 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be716b37-9e6d-4ca5-955d-def565889a13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:38.651964 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be716b37-9e6d-4ca5-955d-def565889a13 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:38.752905 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1017/2023] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 08:57:38 2022] POST /identity/v3/projects => generated 369 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:57:38.756882 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39715e3-6253-4085-9d58-b8084302ddca admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:38.767465 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39715e3-6253-4085-9d58-b8084302ddca admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:38.837092 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d1d590-4574-4053-aded-48eb733b8917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:38.837728 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d1d590-4574-4053-aded-48eb733b8917 admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:38.838006 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d1d590-4574-4053-aded-48eb733b8917 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:38.838278 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d1d590-4574-4053-aded-48eb733b8917 admin admin] PATH_INFO: `/v3/users` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:38.838920 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d1d590-4574-4053-aded-48eb733b8917 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:38.840968 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d1d590-4574-4053-aded-48eb733b8917 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:38.964126 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1007/2024] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 08:57:38 2022] POST /identity/v3/users => generated 285 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:57:38.969891 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be716b37-9e6d-4ca5-955d-def565889a13 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:38.983683 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be716b37-9e6d-4ca5-955d-def565889a13 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:38.988857 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e7229a-c65b-4cf2-a30d-b40d5b999a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:38.989495 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e7229a-c65b-4cf2-a30d-b40d5b999a0f admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:38.989767 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e7229a-c65b-4cf2-a30d-b40d5b999a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:38.990037 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e7229a-c65b-4cf2-a30d-b40d5b999a0f admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:38.990716 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e7229a-c65b-4cf2-a30d-b40d5b999a0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:38.992459 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e7229a-c65b-4cf2-a30d-b40d5b999a0f admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:39.004017 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1018/2025] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:57:38 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:57:39.011234 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d1d590-4574-4053-aded-48eb733b8917 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:39.022742 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d1d590-4574-4053-aded-48eb733b8917 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:39.027943 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1cdca0-3655-4c3c-9540-42be591dbc51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:39.028722 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1cdca0-3655-4c3c-9540-42be591dbc51 admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:39.028999 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1cdca0-3655-4c3c-9540-42be591dbc51 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:39.030170 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1cdca0-3655-4c3c-9540-42be591dbc51 admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:39.030457 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1cdca0-3655-4c3c-9540-42be591dbc51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:39.032095 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1cdca0-3655-4c3c-9540-42be591dbc51 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:39.063348 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1008/2026] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:57:39 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:57:39.067359 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e7229a-c65b-4cf2-a30d-b40d5b999a0f admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:39.078912 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e7229a-c65b-4cf2-a30d-b40d5b999a0f admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:39.085028 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12239037-42d2-465f-99f1-75fff9638933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:39.088288 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12239037-42d2-465f-99f1-75fff9638933 admin admin] REQUEST_METHOD: `PUT` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:39.088288 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12239037-42d2-465f-99f1-75fff9638933 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:39.088288 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12239037-42d2-465f-99f1-75fff9638933 admin admin] PATH_INFO: `/v3/projects/4eaa8c7faf184da98fb86fcdc660cec0/users/210acaec4e5843338b05e7c2e0d00f49/roles/b674fd0a592e4e0cada811bba5ceea4d` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:39.105299 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12239037-42d2-465f-99f1-75fff9638933 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4eaa8c7faf184da98fb86fcdc660cec0, user_id=210acaec4e5843338b05e7c2e0d00f49, role_id=b674fd0a592e4e0cada811bba5ceea4d)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:39.109288 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12239037-42d2-465f-99f1-75fff9638933 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:39.211074 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1019/2027] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 08:57:39 2022] PUT /identity/v3/projects/4eaa8c7faf184da98fb86fcdc660cec0/users/210acaec4e5843338b05e7c2e0d00f49/roles/b674fd0a592e4e0cada811bba5ceea4d => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:57:39.230105 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1919fd5-a411-4f97-8a9c-f07bb6b7f954 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:39.230612 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1919fd5-a411-4f97-8a9c-f07bb6b7f954 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:39.231131 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1919fd5-a411-4f97-8a9c-f07bb6b7f954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:39.270839 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-e1919fd5-a411-4f97-8a9c-f07bb6b7f954 None None] MFA Rules not processed for user `210acaec4e5843338b05e7c2e0d00f49`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:39.272223 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-e1919fd5-a411-4f97-8a9c-f07bb6b7f954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:39.362460 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1009/2028] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:57:39 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:57:39.367473 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12239037-42d2-465f-99f1-75fff9638933 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:39.452855 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12239037-42d2-465f-99f1-75fff9638933 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:39.460515 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05323a8d-442a-4872-bb55-b4be68f90cc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:39.460515 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05323a8d-442a-4872-bb55-b4be68f90cc2 admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:39.460515 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05323a8d-442a-4872-bb55-b4be68f90cc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:39.461519 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05323a8d-442a-4872-bb55-b4be68f90cc2 admin admin] PATH_INFO: `/v3/projects` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:39.462039 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05323a8d-442a-4872-bb55-b4be68f90cc2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:39.464220 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05323a8d-442a-4872-bb55-b4be68f90cc2 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:39.628147 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1020/2029] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 08:57:39 2022] POST /identity/v3/projects => generated 367 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:57:39.633121 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1919fd5-a411-4f97-8a9c-f07bb6b7f954 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:39.654713 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1919fd5-a411-4f97-8a9c-f07bb6b7f954 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:39.738193 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:39.774547 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a946462-abd9-45f6-8b16-3e297b47a399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:39.775628 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a946462-abd9-45f6-8b16-3e297b47a399 admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:39.776126 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a946462-abd9-45f6-8b16-3e297b47a399 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:39.776777 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a946462-abd9-45f6-8b16-3e297b47a399 admin admin] PATH_INFO: `/v3/users` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:39.777922 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a946462-abd9-45f6-8b16-3e297b47a399 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:39.782176 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a946462-abd9-45f6-8b16-3e297b47a399 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:39.948935 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1010/2030] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 08:57:39 2022] POST /identity/v3/users => generated 290 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:57:39.953200 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05323a8d-442a-4872-bb55-b4be68f90cc2 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:39.964607 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05323a8d-442a-4872-bb55-b4be68f90cc2 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:39.969679 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f926a327-712b-40cf-b73b-6ac3ef1f233c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:39.970343 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f926a327-712b-40cf-b73b-6ac3ef1f233c admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:39.970625 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f926a327-712b-40cf-b73b-6ac3ef1f233c admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:39.970901 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f926a327-712b-40cf-b73b-6ac3ef1f233c admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:39.972081 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f926a327-712b-40cf-b73b-6ac3ef1f233c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:39.973772 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f926a327-712b-40cf-b73b-6ac3ef1f233c admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:39.993100 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1021/2031] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:57:39 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:57:40.001817 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a946462-abd9-45f6-8b16-3e297b47a399 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:40.014031 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a946462-abd9-45f6-8b16-3e297b47a399 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:40.022024 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517f740a-bf11-4cf6-81a2-a10a516e5619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:40.022829 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517f740a-bf11-4cf6-81a2-a10a516e5619 admin admin] REQUEST_METHOD: `PUT` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:40.023189 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517f740a-bf11-4cf6-81a2-a10a516e5619 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:40.023536 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517f740a-bf11-4cf6-81a2-a10a516e5619 admin admin] PATH_INFO: `/v3/projects/968874fd8fd2425ea0a4de53d199ee5b/users/c647bed4294d43ffa2ef8dc013f927ee/roles/b870cd1acaa744e1ae1d58b9ac283821` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:40.052630 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517f740a-bf11-4cf6-81a2-a10a516e5619 admin admin] RBAC: Authorizing `identity:create_grant(project_id=968874fd8fd2425ea0a4de53d199ee5b, user_id=c647bed4294d43ffa2ef8dc013f927ee, role_id=b870cd1acaa744e1ae1d58b9ac283821)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:40.056573 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517f740a-bf11-4cf6-81a2-a10a516e5619 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:40.139609 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1011/2032] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 08:57:40 2022] PUT /identity/v3/projects/968874fd8fd2425ea0a4de53d199ee5b/users/c647bed4294d43ffa2ef8dc013f927ee/roles/b870cd1acaa744e1ae1d58b9ac283821 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:57:40.143671 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f926a327-712b-40cf-b73b-6ac3ef1f233c admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:40.155202 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f926a327-712b-40cf-b73b-6ac3ef1f233c admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:40.243139 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:40.256796 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b99a240-a345-4532-8830-14686b3353fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:40.257710 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b99a240-a345-4532-8830-14686b3353fd admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:40.258063 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b99a240-a345-4532-8830-14686b3353fd admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:40.259402 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b99a240-a345-4532-8830-14686b3353fd admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:40.259402 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b99a240-a345-4532-8830-14686b3353fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:40.261020 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b99a240-a345-4532-8830-14686b3353fd admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:40.274964 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1022/2033] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:57:40 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 08:57:40.279810 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517f740a-bf11-4cf6-81a2-a10a516e5619 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:40.294140 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517f740a-bf11-4cf6-81a2-a10a516e5619 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:40.301624 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fde1d34-4bf6-4c5b-8c13-dbc9743897ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:40.307354 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fde1d34-4bf6-4c5b-8c13-dbc9743897ab admin admin] REQUEST_METHOD: `PUT` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:40.307354 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fde1d34-4bf6-4c5b-8c13-dbc9743897ab admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:40.307354 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fde1d34-4bf6-4c5b-8c13-dbc9743897ab admin admin] PATH_INFO: `/v3/projects/968874fd8fd2425ea0a4de53d199ee5b/users/c647bed4294d43ffa2ef8dc013f927ee/roles/b870cd1acaa744e1ae1d58b9ac283821` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:40.307354 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fde1d34-4bf6-4c5b-8c13-dbc9743897ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=968874fd8fd2425ea0a4de53d199ee5b, user_id=c647bed4294d43ffa2ef8dc013f927ee, role_id=b870cd1acaa744e1ae1d58b9ac283821)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:40.307996 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fde1d34-4bf6-4c5b-8c13-dbc9743897ab admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:40.324688 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1012/2034] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 08:57:40 2022] PUT /identity/v3/projects/968874fd8fd2425ea0a4de53d199ee5b/users/c647bed4294d43ffa2ef8dc013f927ee/roles/b870cd1acaa744e1ae1d58b9ac283821 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:57:40.334266 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbb450c-b53d-491e-a4ca-12e9dc312efe None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:40.339862 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbb450c-b53d-491e-a4ca-12e9dc312efe None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:40.339862 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbb450c-b53d-491e-a4ca-12e9dc312efe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:40.370265 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-cbbb450c-b53d-491e-a4ca-12e9dc312efe None None] MFA Rules not processed for user `c647bed4294d43ffa2ef8dc013f927ee`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:40.371716 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-cbbb450c-b53d-491e-a4ca-12e9dc312efe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:40.493281 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1023/2035] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:57:40 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:57:40.512798 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fde1d34-4bf6-4c5b-8c13-dbc9743897ab admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:40.514866 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-1fde1d34-4bf6-4c5b-8c13-dbc9743897ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:40.615246 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fde1d34-4bf6-4c5b-8c13-dbc9743897ab admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:40.618434 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534974d3-2c31-40a4-9d93-0319c95bf695 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbb9ac667f04c35a63b2689df18efed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:40.619179 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534974d3-2c31-40a4-9d93-0319c95bf695 service cinder] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:40.619519 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534974d3-2c31-40a4-9d93-0319c95bf695 service cinder] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:40.619852 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534974d3-2c31-40a4-9d93-0319c95bf695 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:40.631685 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534974d3-2c31-40a4-9d93-0319c95bf695 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:40.633523 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534974d3-2c31-40a4-9d93-0319c95bf695 service cinder] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:40.645819 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1013/2036] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:57:40 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:57:40.770979 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbb450c-b53d-491e-a4ca-12e9dc312efe None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:40.773368 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-cbbb450c-b53d-491e-a4ca-12e9dc312efe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:40.804697 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbb450c-b53d-491e-a4ca-12e9dc312efe None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:40.810808 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125898e0-6e14-4619-a192-d29681e73f4e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbb9ac667f04c35a63b2689df18efed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:40.811908 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125898e0-6e14-4619-a192-d29681e73f4e service cinder] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:40.812962 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125898e0-6e14-4619-a192-d29681e73f4e service cinder] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:40.813246 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125898e0-6e14-4619-a192-d29681e73f4e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:40.895586 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125898e0-6e14-4619-a192-d29681e73f4e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:40.897375 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125898e0-6e14-4619-a192-d29681e73f4e service cinder] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:40.917897 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1024/2037] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:57:40 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:57:46.776137 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534974d3-2c31-40a4-9d93-0319c95bf695 service cinder] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:46.843523 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534974d3-2c31-40a4-9d93-0319c95bf695 service cinder] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:46.848709 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2c3f60-1eba-42dc-9918-00276962f932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:46.849350 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2c3f60-1eba-42dc-9918-00276962f932 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:46.849635 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2c3f60-1eba-42dc-9918-00276962f932 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:46.849905 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2c3f60-1eba-42dc-9918-00276962f932 admin admin] PATH_INFO: `/v3/users/210acaec4e5843338b05e7c2e0d00f49` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:46.851159 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2c3f60-1eba-42dc-9918-00276962f932 admin admin] RBAC: Authorizing `identity:delete_user(user_id=210acaec4e5843338b05e7c2e0d00f49)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:46.853250 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2c3f60-1eba-42dc-9918-00276962f932 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:47.082051 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:47.089707 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-fe2c3f60-1eba-42dc-9918-00276962f932 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '210acaec4e5843338b05e7c2e0d00f49'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:47.090489 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-fe2c3f60-1eba-42dc-9918-00276962f932 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '210acaec4e5843338b05e7c2e0d00f49'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:47.091184 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-fe2c3f60-1eba-42dc-9918-00276962f932 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '210acaec4e5843338b05e7c2e0d00f49'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:47.106575 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-fe2c3f60-1eba-42dc-9918-00276962f932 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '210acaec4e5843338b05e7c2e0d00f49'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:47.195837 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-fe2c3f60-1eba-42dc-9918-00276962f932 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '210acaec4e5843338b05e7c2e0d00f49'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:47.211386 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1014/2038] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 08:57:46 2022] DELETE /identity/v3/users/210acaec4e5843338b05e7c2e0d00f49 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:57:47.216169 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17568597-8b3c-4c98-a01a-27698ef333ba None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:47.216766 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17568597-8b3c-4c98-a01a-27698ef333ba None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:47.216766 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17568597-8b3c-4c98-a01a-27698ef333ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:47.238398 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-17568597-8b3c-4c98-a01a-27698ef333ba None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:47.239635 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-17568597-8b3c-4c98-a01a-27698ef333ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:47.310390 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1025/2039] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:57:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:57:47.320961 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2c3f60-1eba-42dc-9918-00276962f932 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:47.322418 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-fe2c3f60-1eba-42dc-9918-00276962f932 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:47.394154 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2c3f60-1eba-42dc-9918-00276962f932 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:47.399589 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cef7544-ce27-4b98-b9bb-6ed6c7988447 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:47.400682 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cef7544-ce27-4b98-b9bb-6ed6c7988447 service neutron] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:47.401044 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cef7544-ce27-4b98-b9bb-6ed6c7988447 service neutron] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:47.401453 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cef7544-ce27-4b98-b9bb-6ed6c7988447 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:47.414698 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cef7544-ce27-4b98-b9bb-6ed6c7988447 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:47.416139 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cef7544-ce27-4b98-b9bb-6ed6c7988447 service neutron] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:47.427197 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1015/2040] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:57:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:57:48.023588 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17568597-8b3c-4c98-a01a-27698ef333ba None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:48.025144 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-17568597-8b3c-4c98-a01a-27698ef333ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:48.039080 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17568597-8b3c-4c98-a01a-27698ef333ba None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:48.044284 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3efe7a2-190f-4e64-b71f-7ffdc633bc56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:48.044997 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3efe7a2-190f-4e64-b71f-7ffdc633bc56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:48.050291 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3efe7a2-190f-4e64-b71f-7ffdc633bc56 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:48.050291 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3efe7a2-190f-4e64-b71f-7ffdc633bc56 admin admin] PATH_INFO: `/v3/projects/4eaa8c7faf184da98fb86fcdc660cec0` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:48.050291 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3efe7a2-190f-4e64-b71f-7ffdc633bc56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4eaa8c7faf184da98fb86fcdc660cec0)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:48.050291 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3efe7a2-190f-4e64-b71f-7ffdc633bc56 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:48.058245 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:48.206810 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-a3efe7a2-190f-4e64-b71f-7ffdc633bc56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4eaa8c7faf184da98fb86fcdc660cec0', 'request_id': 'req-a3efe7a2-190f-4e64-b71f-7ffdc633bc56'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:48.213152 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-a3efe7a2-190f-4e64-b71f-7ffdc633bc56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4eaa8c7faf184da98fb86fcdc660cec0', 'request_id': 'req-a3efe7a2-190f-4e64-b71f-7ffdc633bc56'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:48.302394 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-a3efe7a2-190f-4e64-b71f-7ffdc633bc56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4eaa8c7faf184da98fb86fcdc660cec0', 'request_id': 'req-a3efe7a2-190f-4e64-b71f-7ffdc633bc56'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:48.315397 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-a3efe7a2-190f-4e64-b71f-7ffdc633bc56 admin admin] The token cache is being invalidate because project 4eaa8c7faf184da98fb86fcdc660cec0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:57:48.315687 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-a3efe7a2-190f-4e64-b71f-7ffdc633bc56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:48.316350 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-a3efe7a2-190f-4e64-b71f-7ffdc633bc56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:48.317880 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1026/2041] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 08:57:48 2022] DELETE /identity/v3/projects/4eaa8c7faf184da98fb86fcdc660cec0 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:57:48.322003 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cef7544-ce27-4b98-b9bb-6ed6c7988447 service neutron] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:48.332595 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cef7544-ce27-4b98-b9bb-6ed6c7988447 service neutron] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:48.396686 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e329f51-3e4e-468e-9fa7-4d754668675c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:48.397333 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e329f51-3e4e-468e-9fa7-4d754668675c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:48.397609 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e329f51-3e4e-468e-9fa7-4d754668675c admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:48.397875 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e329f51-3e4e-468e-9fa7-4d754668675c admin admin] PATH_INFO: `/v3/users/c647bed4294d43ffa2ef8dc013f927ee` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:48.399023 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e329f51-3e4e-468e-9fa7-4d754668675c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c647bed4294d43ffa2ef8dc013f927ee)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:48.401038 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e329f51-3e4e-468e-9fa7-4d754668675c admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:48.723250 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-7e329f51-3e4e-468e-9fa7-4d754668675c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c647bed4294d43ffa2ef8dc013f927ee'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:48.724164 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-7e329f51-3e4e-468e-9fa7-4d754668675c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c647bed4294d43ffa2ef8dc013f927ee'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:48.724868 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-7e329f51-3e4e-468e-9fa7-4d754668675c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c647bed4294d43ffa2ef8dc013f927ee'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:48.737478 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:48.739987 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-7e329f51-3e4e-468e-9fa7-4d754668675c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c647bed4294d43ffa2ef8dc013f927ee'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:48.915500 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-7e329f51-3e4e-468e-9fa7-4d754668675c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c647bed4294d43ffa2ef8dc013f927ee'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:48.935276 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1016/2042] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 08:57:48 2022] DELETE /identity/v3/users/c647bed4294d43ffa2ef8dc013f927ee => generated 0 bytes in 614 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:57:49.440388 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3efe7a2-190f-4e64-b71f-7ffdc633bc56 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:49.440388 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-a3efe7a2-190f-4e64-b71f-7ffdc633bc56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:49.460872 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3efe7a2-190f-4e64-b71f-7ffdc633bc56 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:49.571272 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd0925a-0414-4028-a968-87fd4515dfa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:49.572027 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd0925a-0414-4028-a968-87fd4515dfa9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:49.572407 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd0925a-0414-4028-a968-87fd4515dfa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:49.572720 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd0925a-0414-4028-a968-87fd4515dfa9 admin admin] PATH_INFO: `/v3/projects/968874fd8fd2425ea0a4de53d199ee5b` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:49.573903 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd0925a-0414-4028-a968-87fd4515dfa9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=968874fd8fd2425ea0a4de53d199ee5b)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:49.575922 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd0925a-0414-4028-a968-87fd4515dfa9 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:49.747906 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-9fd0925a-0414-4028-a968-87fd4515dfa9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '968874fd8fd2425ea0a4de53d199ee5b', 'request_id': 'req-9fd0925a-0414-4028-a968-87fd4515dfa9'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:49.761169 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-9fd0925a-0414-4028-a968-87fd4515dfa9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '968874fd8fd2425ea0a4de53d199ee5b', 'request_id': 'req-9fd0925a-0414-4028-a968-87fd4515dfa9'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:49.856152 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-9fd0925a-0414-4028-a968-87fd4515dfa9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '968874fd8fd2425ea0a4de53d199ee5b', 'request_id': 'req-9fd0925a-0414-4028-a968-87fd4515dfa9'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:49.878802 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-9fd0925a-0414-4028-a968-87fd4515dfa9 admin admin] The token cache is being invalidate because project 968874fd8fd2425ea0a4de53d199ee5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:57:49.880048 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-9fd0925a-0414-4028-a968-87fd4515dfa9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:49.880988 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-9fd0925a-0414-4028-a968-87fd4515dfa9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:49.882599 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1027/2043] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 08:57:49 2022] DELETE /identity/v3/projects/968874fd8fd2425ea0a4de53d199ee5b => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:57:49.889355 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ee30d2-5841-45a2-b391-1817c40e1f6e None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:49.889985 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ee30d2-5841-45a2-b391-1817c40e1f6e None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:49.890351 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ee30d2-5841-45a2-b391-1817c40e1f6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:49.925371 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-25ee30d2-5841-45a2-b391-1817c40e1f6e None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:49.926736 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-25ee30d2-5841-45a2-b391-1817c40e1f6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:50.015299 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1017/2044] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:57:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:57:50.018909 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2cd340-b724-4bec-a2e2-aaa1b0f65eae None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:50.020578 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2cd340-b724-4bec-a2e2-aaa1b0f65eae None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:50.020578 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2cd340-b724-4bec-a2e2-aaa1b0f65eae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:50.058753 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-4d2cd340-b724-4bec-a2e2-aaa1b0f65eae None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:50.060669 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-4d2cd340-b724-4bec-a2e2-aaa1b0f65eae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:50.074737 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1028/2045] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:57:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:57:50.078996 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ee30d2-5841-45a2-b391-1817c40e1f6e None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:50.090032 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ee30d2-5841-45a2-b391-1817c40e1f6e None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:50.096018 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38530e4b-afd9-4f47-9ce0-af8ac5676e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:50.096686 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38530e4b-afd9-4f47-9ce0-af8ac5676e93 admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:50.097097 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38530e4b-afd9-4f47-9ce0-af8ac5676e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:50.097374 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38530e4b-afd9-4f47-9ce0-af8ac5676e93 admin admin] PATH_INFO: `/v3/domains` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:50.098050 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38530e4b-afd9-4f47-9ce0-af8ac5676e93 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:57:50.098352 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38530e4b-afd9-4f47-9ce0-af8ac5676e93 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:50.099988 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38530e4b-afd9-4f47-9ce0-af8ac5676e93 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:50.115920 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1018/2046] 10.222.0.55 () {58 vars in 1153 bytes} [Wed Aug 24 08:57:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:57:50.122756 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2cd340-b724-4bec-a2e2-aaa1b0f65eae None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:50.135122 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2cd340-b724-4bec-a2e2-aaa1b0f65eae None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:50.140161 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f693314-38cf-4b4a-a196-6bc05c6da8d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:50.140813 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f693314-38cf-4b4a-a196-6bc05c6da8d5 admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:50.141097 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f693314-38cf-4b4a-a196-6bc05c6da8d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:50.141372 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f693314-38cf-4b4a-a196-6bc05c6da8d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:50.142013 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f693314-38cf-4b4a-a196-6bc05c6da8d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:50.144020 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f693314-38cf-4b4a-a196-6bc05c6da8d5 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:50.237064 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:50.249566 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1029/2047] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 08:57:50 2022] POST /identity/v3/projects => generated 373 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:57:50.255862 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38530e4b-afd9-4f47-9ce0-af8ac5676e93 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:50.274443 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38530e4b-afd9-4f47-9ce0-af8ac5676e93 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:50.341648 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7fd76a-6528-4eac-b6da-aac11195dbe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:50.342267 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7fd76a-6528-4eac-b6da-aac11195dbe6 admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:50.342547 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7fd76a-6528-4eac-b6da-aac11195dbe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:50.342818 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7fd76a-6528-4eac-b6da-aac11195dbe6 admin admin] PATH_INFO: `/v3/users` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:50.343487 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7fd76a-6528-4eac-b6da-aac11195dbe6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:50.345427 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7fd76a-6528-4eac-b6da-aac11195dbe6 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:50.435550 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1019/2048] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 08:57:50 2022] POST /identity/v3/users => generated 287 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:57:50.441376 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f693314-38cf-4b4a-a196-6bc05c6da8d5 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:50.454029 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f693314-38cf-4b4a-a196-6bc05c6da8d5 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:50.465763 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79d16ec-c939-4d11-8841-a9375621d82a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:50.466388 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79d16ec-c939-4d11-8841-a9375621d82a admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:50.466666 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79d16ec-c939-4d11-8841-a9375621d82a admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:50.466988 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79d16ec-c939-4d11-8841-a9375621d82a admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:50.467636 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79d16ec-c939-4d11-8841-a9375621d82a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:50.469321 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79d16ec-c939-4d11-8841-a9375621d82a admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:50.482238 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1030/2049] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:57:50 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:57:50.487685 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7fd76a-6528-4eac-b6da-aac11195dbe6 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:50.501899 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7fd76a-6528-4eac-b6da-aac11195dbe6 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:50.507137 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66463884-8c21-4cbb-a5d1-85d254eff500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:50.507786 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66463884-8c21-4cbb-a5d1-85d254eff500 admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:50.508117 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66463884-8c21-4cbb-a5d1-85d254eff500 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:50.508430 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66463884-8c21-4cbb-a5d1-85d254eff500 admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:50.509143 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66463884-8c21-4cbb-a5d1-85d254eff500 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:50.510823 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66463884-8c21-4cbb-a5d1-85d254eff500 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:50.523205 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1020/2050] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:57:50 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:57:50.527807 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79d16ec-c939-4d11-8841-a9375621d82a admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:50.539501 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79d16ec-c939-4d11-8841-a9375621d82a admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:50.544436 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c140cb83-39ec-425d-a706-e97d5673ab60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:50.545151 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c140cb83-39ec-425d-a706-e97d5673ab60 admin admin] REQUEST_METHOD: `PUT` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:50.545427 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c140cb83-39ec-425d-a706-e97d5673ab60 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:50.545698 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c140cb83-39ec-425d-a706-e97d5673ab60 admin admin] PATH_INFO: `/v3/projects/bd3ebf9d2a02484c9ea5f1fe959a4618/users/70bb349cf6cc44f385c6434dfaa1b7c6/roles/b674fd0a592e4e0cada811bba5ceea4d` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:50.564456 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c140cb83-39ec-425d-a706-e97d5673ab60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd3ebf9d2a02484c9ea5f1fe959a4618, user_id=70bb349cf6cc44f385c6434dfaa1b7c6, role_id=b674fd0a592e4e0cada811bba5ceea4d)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:50.569959 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c140cb83-39ec-425d-a706-e97d5673ab60 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:50.662971 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1031/2051] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 08:57:50 2022] PUT /identity/v3/projects/bd3ebf9d2a02484c9ea5f1fe959a4618/users/70bb349cf6cc44f385c6434dfaa1b7c6/roles/b674fd0a592e4e0cada811bba5ceea4d => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:57:50.670080 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21b4533-5c6f-4898-8edc-51a2995f738f None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:50.670646 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21b4533-5c6f-4898-8edc-51a2995f738f None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:50.676207 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21b4533-5c6f-4898-8edc-51a2995f738f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:50.697122 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-a21b4533-5c6f-4898-8edc-51a2995f738f None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:50.698360 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-a21b4533-5c6f-4898-8edc-51a2995f738f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:50.754415 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:50.785093 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1021/2052] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:57:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:57:50.793312 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c140cb83-39ec-425d-a706-e97d5673ab60 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:50.795253 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-c140cb83-39ec-425d-a706-e97d5673ab60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:50.865114 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c140cb83-39ec-425d-a706-e97d5673ab60 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:50.869133 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147a1fb3-4ab9-43ae-a65d-0407d083a445 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:50.869808 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147a1fb3-4ab9-43ae-a65d-0407d083a445 service neutron] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:50.870088 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147a1fb3-4ab9-43ae-a65d-0407d083a445 service neutron] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:50.870364 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147a1fb3-4ab9-43ae-a65d-0407d083a445 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:50.880314 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147a1fb3-4ab9-43ae-a65d-0407d083a445 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:50.881812 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147a1fb3-4ab9-43ae-a65d-0407d083a445 service neutron] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:50.893413 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1032/2053] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:57:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:57:52.733985 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9468d476-f9aa-46b8-ac67-b2b3ee095c74 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:52.734746 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9468d476-f9aa-46b8-ac67-b2b3ee095c74 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:52.735099 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9468d476-f9aa-46b8-ac67-b2b3ee095c74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:52.769549 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-9468d476-f9aa-46b8-ac67-b2b3ee095c74 None None] MFA Rules not processed for user `70bb349cf6cc44f385c6434dfaa1b7c6`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:52.770712 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-9468d476-f9aa-46b8-ac67-b2b3ee095c74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:52.854561 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1022/2054] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:57:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:57:52.859982 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147a1fb3-4ab9-43ae-a65d-0407d083a445 service neutron] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:52.870346 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147a1fb3-4ab9-43ae-a65d-0407d083a445 service neutron] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:52.875138 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b635c2-4051-4ac5-93ce-e7880b29e19f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:52.875756 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b635c2-4051-4ac5-93ce-e7880b29e19f admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:52.876040 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b635c2-4051-4ac5-93ce-e7880b29e19f admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:52.876319 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b635c2-4051-4ac5-93ce-e7880b29e19f admin admin] PATH_INFO: `/v3/projects` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:52.877516 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b635c2-4051-4ac5-93ce-e7880b29e19f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:52.879553 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b635c2-4051-4ac5-93ce-e7880b29e19f admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:53.028669 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1033/2055] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 08:57:52 2022] POST /identity/v3/projects => generated 373 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:57:53.033701 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9468d476-f9aa-46b8-ac67-b2b3ee095c74 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:53.045702 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9468d476-f9aa-46b8-ac67-b2b3ee095c74 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:53.114658 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cb4135-c1fc-4816-b9a3-58258b1502d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:53.115269 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cb4135-c1fc-4816-b9a3-58258b1502d7 admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:53.115549 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cb4135-c1fc-4816-b9a3-58258b1502d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:53.115826 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cb4135-c1fc-4816-b9a3-58258b1502d7 admin admin] PATH_INFO: `/v3/users` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:53.116968 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cb4135-c1fc-4816-b9a3-58258b1502d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:53.118657 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cb4135-c1fc-4816-b9a3-58258b1502d7 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:53.203538 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1023/2056] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 08:57:53 2022] POST /identity/v3/users => generated 293 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:57:53.214122 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b635c2-4051-4ac5-93ce-e7880b29e19f admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:53.224858 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b635c2-4051-4ac5-93ce-e7880b29e19f admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:53.229746 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b530735d-de99-4855-825f-f4e7385a972d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:53.230352 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b530735d-de99-4855-825f-f4e7385a972d admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:53.230630 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b530735d-de99-4855-825f-f4e7385a972d admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:53.230929 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b530735d-de99-4855-825f-f4e7385a972d admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:53.231573 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b530735d-de99-4855-825f-f4e7385a972d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:53.233257 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b530735d-de99-4855-825f-f4e7385a972d admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:53.245291 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1034/2057] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:57:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:57:53.249773 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cb4135-c1fc-4816-b9a3-58258b1502d7 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:53.262683 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cb4135-c1fc-4816-b9a3-58258b1502d7 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:53.267436 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcc5d47-abff-4473-bcab-7e538d8d467e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:53.268044 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcc5d47-abff-4473-bcab-7e538d8d467e admin admin] REQUEST_METHOD: `PUT` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:53.268318 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcc5d47-abff-4473-bcab-7e538d8d467e admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:53.268620 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcc5d47-abff-4473-bcab-7e538d8d467e admin admin] PATH_INFO: `/v3/projects/182cfd3e5c574fc090dabe4c67584751/users/de8e2b95aa4a42ef8d90cdf5787b67e8/roles/b870cd1acaa744e1ae1d58b9ac283821` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:53.288359 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcc5d47-abff-4473-bcab-7e538d8d467e admin admin] RBAC: Authorizing `identity:create_grant(project_id=182cfd3e5c574fc090dabe4c67584751, user_id=de8e2b95aa4a42ef8d90cdf5787b67e8, role_id=b870cd1acaa744e1ae1d58b9ac283821)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:53.288359 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcc5d47-abff-4473-bcab-7e538d8d467e admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:53.390792 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1024/2058] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 08:57:53 2022] PUT /identity/v3/projects/182cfd3e5c574fc090dabe4c67584751/users/de8e2b95aa4a42ef8d90cdf5787b67e8/roles/b870cd1acaa744e1ae1d58b9ac283821 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:57:53.395048 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b530735d-de99-4855-825f-f4e7385a972d admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:53.405549 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b530735d-de99-4855-825f-f4e7385a972d admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:53.465095 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e03b4d6-4482-4d22-93d6-874075c9273d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:53.465713 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e03b4d6-4482-4d22-93d6-874075c9273d admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:53.465987 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e03b4d6-4482-4d22-93d6-874075c9273d admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:53.466259 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e03b4d6-4482-4d22-93d6-874075c9273d admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:53.466998 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e03b4d6-4482-4d22-93d6-874075c9273d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:53.468526 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e03b4d6-4482-4d22-93d6-874075c9273d admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:53.483736 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1035/2059] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:57:53 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:57:53.490396 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcc5d47-abff-4473-bcab-7e538d8d467e admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:53.501342 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcc5d47-abff-4473-bcab-7e538d8d467e admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:53.506217 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad56e5c-3fe1-4a7b-a1b4-83a4da1be22b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:53.507064 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad56e5c-3fe1-4a7b-a1b4-83a4da1be22b admin admin] REQUEST_METHOD: `PUT` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:53.507350 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad56e5c-3fe1-4a7b-a1b4-83a4da1be22b admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:53.507628 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad56e5c-3fe1-4a7b-a1b4-83a4da1be22b admin admin] PATH_INFO: `/v3/projects/182cfd3e5c574fc090dabe4c67584751/users/de8e2b95aa4a42ef8d90cdf5787b67e8/roles/b870cd1acaa744e1ae1d58b9ac283821` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:53.509649 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad56e5c-3fe1-4a7b-a1b4-83a4da1be22b admin admin] RBAC: Authorizing `identity:create_grant(project_id=182cfd3e5c574fc090dabe4c67584751, user_id=de8e2b95aa4a42ef8d90cdf5787b67e8, role_id=b870cd1acaa744e1ae1d58b9ac283821)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:53.512380 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad56e5c-3fe1-4a7b-a1b4-83a4da1be22b admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:53.523775 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1025/2060] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 08:57:53 2022] PUT /identity/v3/projects/182cfd3e5c574fc090dabe4c67584751/users/de8e2b95aa4a42ef8d90cdf5787b67e8/roles/b870cd1acaa744e1ae1d58b9ac283821 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:57:55.193124 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa1045b-3efc-47a2-98b9-2ec711979758 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:55.194386 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa1045b-3efc-47a2-98b9-2ec711979758 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:55.195082 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa1045b-3efc-47a2-98b9-2ec711979758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:55.235748 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-caa1045b-3efc-47a2-98b9-2ec711979758 None None] MFA Rules not processed for user `de8e2b95aa4a42ef8d90cdf5787b67e8`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:55.237198 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-caa1045b-3efc-47a2-98b9-2ec711979758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:55.259635 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:55.331104 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1036/2061] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:57:55 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:57:55.381403 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad56e5c-3fe1-4a7b-a1b4-83a4da1be22b admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:55.383660 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-5ad56e5c-3fe1-4a7b-a1b4-83a4da1be22b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:55.421620 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:55.472967 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad56e5c-3fe1-4a7b-a1b4-83a4da1be22b admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:55.479227 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d036d9-e750-4040-bcb6-309457a68d5d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbb9ac667f04c35a63b2689df18efed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:55.483419 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d036d9-e750-4040-bcb6-309457a68d5d service cinder] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:55.483419 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d036d9-e750-4040-bcb6-309457a68d5d service cinder] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:55.483419 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d036d9-e750-4040-bcb6-309457a68d5d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:55.571893 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d036d9-e750-4040-bcb6-309457a68d5d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:55.574764 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d036d9-e750-4040-bcb6-309457a68d5d service cinder] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:55.593024 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1026/2062] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:57:55 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:57:57.473892 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa1045b-3efc-47a2-98b9-2ec711979758 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:57.475553 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-caa1045b-3efc-47a2-98b9-2ec711979758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:57.571619 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa1045b-3efc-47a2-98b9-2ec711979758 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:57.576964 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c231c6a-4c5f-4fe7-975d-6c14e4e359a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:57.578155 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c231c6a-4c5f-4fe7-975d-6c14e4e359a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:57.579181 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c231c6a-4c5f-4fe7-975d-6c14e4e359a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:57.579664 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c231c6a-4c5f-4fe7-975d-6c14e4e359a3 admin admin] PATH_INFO: `/v3/users/315fd1ff671a48e2945289e247782ae1` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:57.581497 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c231c6a-4c5f-4fe7-975d-6c14e4e359a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=315fd1ff671a48e2945289e247782ae1)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:57.584046 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c231c6a-4c5f-4fe7-975d-6c14e4e359a3 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:57.823279 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-3c231c6a-4c5f-4fe7-975d-6c14e4e359a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '315fd1ff671a48e2945289e247782ae1'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:57.824209 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-3c231c6a-4c5f-4fe7-975d-6c14e4e359a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '315fd1ff671a48e2945289e247782ae1'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:57.824936 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-3c231c6a-4c5f-4fe7-975d-6c14e4e359a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '315fd1ff671a48e2945289e247782ae1'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:57.839666 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-3c231c6a-4c5f-4fe7-975d-6c14e4e359a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '315fd1ff671a48e2945289e247782ae1'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:57.933335 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-3c231c6a-4c5f-4fe7-975d-6c14e4e359a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '315fd1ff671a48e2945289e247782ae1'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:57.957818 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1037/2063] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 08:57:57 2022] DELETE /identity/v3/users/315fd1ff671a48e2945289e247782ae1 => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:57:57.962308 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541a2818-595d-4e34-aa4d-50f45d7d22e8 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:57.962914 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541a2818-595d-4e34-aa4d-50f45d7d22e8 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:57.963173 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541a2818-595d-4e34-aa4d-50f45d7d22e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:57.990479 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-541a2818-595d-4e34-aa4d-50f45d7d22e8 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:57.992967 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-541a2818-595d-4e34-aa4d-50f45d7d22e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:58.069241 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1027/2064] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:57:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:57:58.079303 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c231c6a-4c5f-4fe7-975d-6c14e4e359a3 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:58.080884 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-3c231c6a-4c5f-4fe7-975d-6c14e4e359a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:58.120964 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:58.147468 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c231c6a-4c5f-4fe7-975d-6c14e4e359a3 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:58.152089 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e167c5e-e9f9-421f-a334-cb5aace4ea24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:58.152763 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e167c5e-e9f9-421f-a334-cb5aace4ea24 service neutron] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:58.153037 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e167c5e-e9f9-421f-a334-cb5aace4ea24 service neutron] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:58.153313 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e167c5e-e9f9-421f-a334-cb5aace4ea24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:58.164964 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e167c5e-e9f9-421f-a334-cb5aace4ea24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:58.166761 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e167c5e-e9f9-421f-a334-cb5aace4ea24 service neutron] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:58.178536 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1038/2065] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:57:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:57:58.648182 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541a2818-595d-4e34-aa4d-50f45d7d22e8 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:58.649976 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-541a2818-595d-4e34-aa4d-50f45d7d22e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:58.704888 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e167c5e-e9f9-421f-a334-cb5aace4ea24 service neutron] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:58.711227 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-9e167c5e-e9f9-421f-a334-cb5aace4ea24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:58.723715 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541a2818-595d-4e34-aa4d-50f45d7d22e8 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:58.725543 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e167c5e-e9f9-421f-a334-cb5aace4ea24 service neutron] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:58.729453 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b930678-ef25-42ca-bedf-f1798fb8459e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbb9ac667f04c35a63b2689df18efed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:58.729975 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b930678-ef25-42ca-bedf-f1798fb8459e service cinder] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:58.730546 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae166c38-f49b-49f9-aaf6-5dbe2a752cdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:58.730689 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b930678-ef25-42ca-bedf-f1798fb8459e service cinder] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:58.731024 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b930678-ef25-42ca-bedf-f1798fb8459e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:58.731192 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae166c38-f49b-49f9-aaf6-5dbe2a752cdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:58.731479 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae166c38-f49b-49f9-aaf6-5dbe2a752cdd admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:58.731917 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae166c38-f49b-49f9-aaf6-5dbe2a752cdd admin admin] PATH_INFO: `/v3/projects/81db7fdc34dc4964a383c1b22e757934` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:58.733224 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae166c38-f49b-49f9-aaf6-5dbe2a752cdd admin admin] RBAC: Authorizing `identity:delete_project(project_id=81db7fdc34dc4964a383c1b22e757934)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:58.734480 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-4b930678-ef25-42ca-bedf-f1798fb8459e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:58.735427 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae166c38-f49b-49f9-aaf6-5dbe2a752cdd admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:58.799169 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:58.854713 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b930678-ef25-42ca-bedf-f1798fb8459e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:58.856694 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b930678-ef25-42ca-bedf-f1798fb8459e service cinder] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:58.877211 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1028/2066] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:57:58 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:57:58.878145 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-ae166c38-f49b-49f9-aaf6-5dbe2a752cdd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81db7fdc34dc4964a383c1b22e757934', 'request_id': 'req-ae166c38-f49b-49f9-aaf6-5dbe2a752cdd'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:58.892468 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-ae166c38-f49b-49f9-aaf6-5dbe2a752cdd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81db7fdc34dc4964a383c1b22e757934', 'request_id': 'req-ae166c38-f49b-49f9-aaf6-5dbe2a752cdd'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:58.984266 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-ae166c38-f49b-49f9-aaf6-5dbe2a752cdd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81db7fdc34dc4964a383c1b22e757934', 'request_id': 'req-ae166c38-f49b-49f9-aaf6-5dbe2a752cdd'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:58.998374 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-ae166c38-f49b-49f9-aaf6-5dbe2a752cdd admin admin] The token cache is being invalidate because project 81db7fdc34dc4964a383c1b22e757934 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:57:58.998667 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-ae166c38-f49b-49f9-aaf6-5dbe2a752cdd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:58.999327 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-ae166c38-f49b-49f9-aaf6-5dbe2a752cdd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:59.000893 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1039/2067] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 08:57:58 2022] DELETE /identity/v3/projects/81db7fdc34dc4964a383c1b22e757934 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:57:59.008827 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7e85d7-2323-491d-8e92-ee87732d95f3 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:59.009562 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7e85d7-2323-491d-8e92-ee87732d95f3 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:59.009830 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7e85d7-2323-491d-8e92-ee87732d95f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:59.038544 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-3c7e85d7-2323-491d-8e92-ee87732d95f3 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:59.040714 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-3c7e85d7-2323-491d-8e92-ee87732d95f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:59.138223 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1029/2068] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:57:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:57:59.147083 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab24e962-9e72-4e6a-b8cb-57b6a139604f None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:59.147374 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab24e962-9e72-4e6a-b8cb-57b6a139604f None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:59.147648 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab24e962-9e72-4e6a-b8cb-57b6a139604f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:59.169837 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-ab24e962-9e72-4e6a-b8cb-57b6a139604f None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:59.171315 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-ab24e962-9e72-4e6a-b8cb-57b6a139604f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:59.185187 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1040/2069] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:57:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:57:59.196513 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7e85d7-2323-491d-8e92-ee87732d95f3 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:59.208424 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7e85d7-2323-491d-8e92-ee87732d95f3 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:59.216416 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe16077f-ee02-47f9-accb-a627eb2487f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:59.217101 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe16077f-ee02-47f9-accb-a627eb2487f7 admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:59.217451 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe16077f-ee02-47f9-accb-a627eb2487f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:59.217793 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe16077f-ee02-47f9-accb-a627eb2487f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:59.218873 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe16077f-ee02-47f9-accb-a627eb2487f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:57:59.218873 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe16077f-ee02-47f9-accb-a627eb2487f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:59.221277 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe16077f-ee02-47f9-accb-a627eb2487f7 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:59.239099 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1030/2070] 10.222.0.55 () {58 vars in 1153 bytes} [Wed Aug 24 08:57:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:57:59.243371 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab24e962-9e72-4e6a-b8cb-57b6a139604f None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:59.257695 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab24e962-9e72-4e6a-b8cb-57b6a139604f None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:59.263127 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4dea23-8595-4a5c-820d-23d81bd8a463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:59.263767 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4dea23-8595-4a5c-820d-23d81bd8a463 admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:59.264045 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4dea23-8595-4a5c-820d-23d81bd8a463 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:59.264318 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4dea23-8595-4a5c-820d-23d81bd8a463 admin admin] PATH_INFO: `/v3/projects` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:59.265101 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4dea23-8595-4a5c-820d-23d81bd8a463 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:59.267580 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4dea23-8595-4a5c-820d-23d81bd8a463 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:59.366800 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1041/2071] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 08:57:59 2022] POST /identity/v3/projects => generated 375 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:57:59.371075 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe16077f-ee02-47f9-accb-a627eb2487f7 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:59.382365 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe16077f-ee02-47f9-accb-a627eb2487f7 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:59.476605 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b26c95f-2f0f-4d2b-b7df-ba8a4a6cd7c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:59.477243 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b26c95f-2f0f-4d2b-b7df-ba8a4a6cd7c3 admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:59.477519 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b26c95f-2f0f-4d2b-b7df-ba8a4a6cd7c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:59.477789 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b26c95f-2f0f-4d2b-b7df-ba8a4a6cd7c3 admin admin] PATH_INFO: `/v3/users` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:59.478453 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b26c95f-2f0f-4d2b-b7df-ba8a4a6cd7c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:59.480691 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b26c95f-2f0f-4d2b-b7df-ba8a4a6cd7c3 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:59.585547 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1031/2072] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 08:57:59 2022] POST /identity/v3/users => generated 288 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:57:59.590552 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4dea23-8595-4a5c-820d-23d81bd8a463 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:59.603040 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4dea23-8595-4a5c-820d-23d81bd8a463 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:59.608228 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86df0b9c-fa9a-43bd-a8f3-d6b3cb3dae9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:59.608916 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86df0b9c-fa9a-43bd-a8f3-d6b3cb3dae9c admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:59.609196 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86df0b9c-fa9a-43bd-a8f3-d6b3cb3dae9c admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:59.609467 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86df0b9c-fa9a-43bd-a8f3-d6b3cb3dae9c admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:59.610122 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86df0b9c-fa9a-43bd-a8f3-d6b3cb3dae9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:59.612355 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86df0b9c-fa9a-43bd-a8f3-d6b3cb3dae9c admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:59.624477 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1042/2073] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:57:59 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:57:59.628532 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b26c95f-2f0f-4d2b-b7df-ba8a4a6cd7c3 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:59.639520 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b26c95f-2f0f-4d2b-b7df-ba8a4a6cd7c3 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:59.644575 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49868fd8-85ca-4a4f-98d9-228dc3c007a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:59.645202 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49868fd8-85ca-4a4f-98d9-228dc3c007a5 admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:59.645479 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49868fd8-85ca-4a4f-98d9-228dc3c007a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:59.645749 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49868fd8-85ca-4a4f-98d9-228dc3c007a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:59.646398 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49868fd8-85ca-4a4f-98d9-228dc3c007a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:59.648008 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49868fd8-85ca-4a4f-98d9-228dc3c007a5 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:59.661206 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1032/2074] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:57:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:57:59.666579 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86df0b9c-fa9a-43bd-a8f3-d6b3cb3dae9c admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:59.679114 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86df0b9c-fa9a-43bd-a8f3-d6b3cb3dae9c admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:59.684179 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cf4b70-b815-4a8f-b3a1-b3bfa16a765b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:59.684924 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cf4b70-b815-4a8f-b3a1-b3bfa16a765b admin admin] REQUEST_METHOD: `PUT` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:59.685218 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cf4b70-b815-4a8f-b3a1-b3bfa16a765b admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:59.685494 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cf4b70-b815-4a8f-b3a1-b3bfa16a765b admin admin] PATH_INFO: `/v3/projects/037efe7276854e73a080b845ca1c4c04/users/31050a52d1954370ba1a9be073212401/roles/b674fd0a592e4e0cada811bba5ceea4d` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:59.702177 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cf4b70-b815-4a8f-b3a1-b3bfa16a765b admin admin] RBAC: Authorizing `identity:create_grant(project_id=037efe7276854e73a080b845ca1c4c04, user_id=31050a52d1954370ba1a9be073212401, role_id=b674fd0a592e4e0cada811bba5ceea4d)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:59.705614 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cf4b70-b815-4a8f-b3a1-b3bfa16a765b admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:59.846734 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1043/2075] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 08:57:59 2022] PUT /identity/v3/projects/037efe7276854e73a080b845ca1c4c04/users/31050a52d1954370ba1a9be073212401/roles/b674fd0a592e4e0cada811bba5ceea4d => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:57:59.857725 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9237cca3-8412-4731-bdf1-7506374cac88 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:59.858079 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9237cca3-8412-4731-bdf1-7506374cac88 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:59.858405 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9237cca3-8412-4731-bdf1-7506374cac88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:59.898186 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-9237cca3-8412-4731-bdf1-7506374cac88 None None] MFA Rules not processed for user `31050a52d1954370ba1a9be073212401`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:59.899662 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-9237cca3-8412-4731-bdf1-7506374cac88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:59.946235 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:00.005463 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1033/2076] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:57:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:58:00.012157 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cf4b70-b815-4a8f-b3a1-b3bfa16a765b admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:00.115422 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:00.121644 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cf4b70-b815-4a8f-b3a1-b3bfa16a765b admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:00.132106 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b196ae-263a-451b-8770-87e0dd5726b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:00.133392 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b196ae-263a-451b-8770-87e0dd5726b7 admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:00.133799 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b196ae-263a-451b-8770-87e0dd5726b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:00.134162 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b196ae-263a-451b-8770-87e0dd5726b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:00.134922 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b196ae-263a-451b-8770-87e0dd5726b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:00.137618 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b196ae-263a-451b-8770-87e0dd5726b7 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:00.245790 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1044/2077] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 08:58:00 2022] POST /identity/v3/projects => generated 375 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:58:00.250770 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9237cca3-8412-4731-bdf1-7506374cac88 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:00.274388 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9237cca3-8412-4731-bdf1-7506374cac88 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:00.363944 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a8783d-9083-4081-8efd-dadbd6f29822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:00.365018 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a8783d-9083-4081-8efd-dadbd6f29822 admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:00.365521 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a8783d-9083-4081-8efd-dadbd6f29822 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:00.366023 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a8783d-9083-4081-8efd-dadbd6f29822 admin admin] PATH_INFO: `/v3/users` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:00.367258 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a8783d-9083-4081-8efd-dadbd6f29822 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:00.371310 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a8783d-9083-4081-8efd-dadbd6f29822 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:00.506465 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1034/2078] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 08:58:00 2022] POST /identity/v3/users => generated 288 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:58:00.509826 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b196ae-263a-451b-8770-87e0dd5726b7 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:00.541747 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b196ae-263a-451b-8770-87e0dd5726b7 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:00.552037 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661f7d77-48fb-42b2-872f-b534f3e3d3bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:00.552721 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661f7d77-48fb-42b2-872f-b534f3e3d3bb admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:00.553065 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661f7d77-48fb-42b2-872f-b534f3e3d3bb admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:00.553285 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661f7d77-48fb-42b2-872f-b534f3e3d3bb admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:00.554216 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661f7d77-48fb-42b2-872f-b534f3e3d3bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:00.555890 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661f7d77-48fb-42b2-872f-b534f3e3d3bb admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:00.573138 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1045/2079] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:58:00 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:58:00.574767 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a8783d-9083-4081-8efd-dadbd6f29822 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:00.588966 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a8783d-9083-4081-8efd-dadbd6f29822 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:00.597153 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03342983-2a01-4b95-ab75-ebd13f2428ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:00.597792 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03342983-2a01-4b95-ab75-ebd13f2428ce admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:00.598073 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03342983-2a01-4b95-ab75-ebd13f2428ce admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:00.598366 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03342983-2a01-4b95-ab75-ebd13f2428ce admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:00.599112 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03342983-2a01-4b95-ab75-ebd13f2428ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:00.601100 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03342983-2a01-4b95-ab75-ebd13f2428ce admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:00.621891 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1035/2080] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:58:00 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:58:00.625805 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661f7d77-48fb-42b2-872f-b534f3e3d3bb admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:00.639849 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661f7d77-48fb-42b2-872f-b534f3e3d3bb admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:00.665208 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f305e399-dccb-4af1-bbb6-deb7d05b5b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:00.666080 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f305e399-dccb-4af1-bbb6-deb7d05b5b97 admin admin] REQUEST_METHOD: `PUT` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:00.666459 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f305e399-dccb-4af1-bbb6-deb7d05b5b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:00.666873 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f305e399-dccb-4af1-bbb6-deb7d05b5b97 admin admin] PATH_INFO: `/v3/projects/f9d0d2e9249d47169531aaa5ee062621/users/a56c3f5859484ea28194a157570570f7/roles/b674fd0a592e4e0cada811bba5ceea4d` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:00.689910 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f305e399-dccb-4af1-bbb6-deb7d05b5b97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9d0d2e9249d47169531aaa5ee062621, user_id=a56c3f5859484ea28194a157570570f7, role_id=b674fd0a592e4e0cada811bba5ceea4d)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:00.692861 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f305e399-dccb-4af1-bbb6-deb7d05b5b97 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:00.801421 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1046/2081] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 08:58:00 2022] PUT /identity/v3/projects/f9d0d2e9249d47169531aaa5ee062621/users/a56c3f5859484ea28194a157570570f7/roles/b674fd0a592e4e0cada811bba5ceea4d => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:58:00.813198 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93fa2d4-0634-4c2e-bb13-30441369ac67 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:00.813484 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93fa2d4-0634-4c2e-bb13-30441369ac67 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:00.813759 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93fa2d4-0634-4c2e-bb13-30441369ac67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:00.852747 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-e93fa2d4-0634-4c2e-bb13-30441369ac67 None None] MFA Rules not processed for user `a56c3f5859484ea28194a157570570f7`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:00.854948 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-e93fa2d4-0634-4c2e-bb13-30441369ac67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:00.992559 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1036/2082] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:58:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:58:00.997870 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f305e399-dccb-4af1-bbb6-deb7d05b5b97 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:01.069698 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f305e399-dccb-4af1-bbb6-deb7d05b5b97 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:01.075315 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c51252-af2e-4fa8-957d-450c43352bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:01.077808 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c51252-af2e-4fa8-957d-450c43352bc8 admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:01.077808 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c51252-af2e-4fa8-957d-450c43352bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:01.077808 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c51252-af2e-4fa8-957d-450c43352bc8 admin admin] PATH_INFO: `/v3/projects` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:01.077808 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c51252-af2e-4fa8-957d-450c43352bc8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:01.079228 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c51252-af2e-4fa8-957d-450c43352bc8 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:01.231376 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1047/2083] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 08:58:00 2022] POST /identity/v3/projects => generated 377 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:58:01.235147 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93fa2d4-0634-4c2e-bb13-30441369ac67 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:01.247050 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93fa2d4-0634-4c2e-bb13-30441369ac67 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:01.304352 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:01.341866 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88ba2b2-aaa4-4450-8cfb-27c27f9e5aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:01.342522 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88ba2b2-aaa4-4450-8cfb-27c27f9e5aae admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:01.342804 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88ba2b2-aaa4-4450-8cfb-27c27f9e5aae admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:01.343078 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88ba2b2-aaa4-4450-8cfb-27c27f9e5aae admin admin] PATH_INFO: `/v3/users` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:01.343730 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88ba2b2-aaa4-4450-8cfb-27c27f9e5aae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:01.345754 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88ba2b2-aaa4-4450-8cfb-27c27f9e5aae admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:01.486161 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1037/2084] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 08:58:01 2022] POST /identity/v3/users => generated 295 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:58:01.491184 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c51252-af2e-4fa8-957d-450c43352bc8 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:01.511733 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c51252-af2e-4fa8-957d-450c43352bc8 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:01.520022 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f6bb48-6c17-4bb6-be03-3accc304645a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:01.527659 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f6bb48-6c17-4bb6-be03-3accc304645a admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:01.527659 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f6bb48-6c17-4bb6-be03-3accc304645a admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:01.527659 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f6bb48-6c17-4bb6-be03-3accc304645a admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:01.527659 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f6bb48-6c17-4bb6-be03-3accc304645a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:01.527659 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f6bb48-6c17-4bb6-be03-3accc304645a admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:01.536376 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1048/2085] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:58:01 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:58:01.542974 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88ba2b2-aaa4-4450-8cfb-27c27f9e5aae admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:01.563895 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88ba2b2-aaa4-4450-8cfb-27c27f9e5aae admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:01.573579 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41104d6-e0aa-4d22-9b4e-7d951357995c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:01.574936 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41104d6-e0aa-4d22-9b4e-7d951357995c admin admin] REQUEST_METHOD: `PUT` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:01.575464 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41104d6-e0aa-4d22-9b4e-7d951357995c admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:01.576075 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41104d6-e0aa-4d22-9b4e-7d951357995c admin admin] PATH_INFO: `/v3/projects/e7a58a430a394ef280ccb2ca1c17a1e1/users/520fe6cbf9464583a38c6dabd0b86f8c/roles/b870cd1acaa744e1ae1d58b9ac283821` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:01.601515 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41104d6-e0aa-4d22-9b4e-7d951357995c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7a58a430a394ef280ccb2ca1c17a1e1, user_id=520fe6cbf9464583a38c6dabd0b86f8c, role_id=b870cd1acaa744e1ae1d58b9ac283821)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:01.604256 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41104d6-e0aa-4d22-9b4e-7d951357995c admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:01.713258 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1038/2086] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 08:58:01 2022] PUT /identity/v3/projects/e7a58a430a394ef280ccb2ca1c17a1e1/users/520fe6cbf9464583a38c6dabd0b86f8c/roles/b870cd1acaa744e1ae1d58b9ac283821 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:58:01.727088 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f6bb48-6c17-4bb6-be03-3accc304645a admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:01.739139 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f6bb48-6c17-4bb6-be03-3accc304645a admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:01.816164 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61745652-0b76-4fb1-bc07-b43dd0b00582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:01.817283 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61745652-0b76-4fb1-bc07-b43dd0b00582 admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:01.817576 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61745652-0b76-4fb1-bc07-b43dd0b00582 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:01.817857 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61745652-0b76-4fb1-bc07-b43dd0b00582 admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:01.818687 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61745652-0b76-4fb1-bc07-b43dd0b00582 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:01.820326 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61745652-0b76-4fb1-bc07-b43dd0b00582 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:01.831735 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1049/2087] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:58:01 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:58:01.837848 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41104d6-e0aa-4d22-9b4e-7d951357995c admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:01.847839 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41104d6-e0aa-4d22-9b4e-7d951357995c admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:01.853315 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776dc02b-f1f4-4736-8350-7d8c7a1cccfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:01.854118 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776dc02b-f1f4-4736-8350-7d8c7a1cccfa admin admin] REQUEST_METHOD: `PUT` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:01.854517 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776dc02b-f1f4-4736-8350-7d8c7a1cccfa admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:01.854889 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776dc02b-f1f4-4736-8350-7d8c7a1cccfa admin admin] PATH_INFO: `/v3/projects/e7a58a430a394ef280ccb2ca1c17a1e1/users/520fe6cbf9464583a38c6dabd0b86f8c/roles/b870cd1acaa744e1ae1d58b9ac283821` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:01.857021 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776dc02b-f1f4-4736-8350-7d8c7a1cccfa admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7a58a430a394ef280ccb2ca1c17a1e1, user_id=520fe6cbf9464583a38c6dabd0b86f8c, role_id=b870cd1acaa744e1ae1d58b9ac283821)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:01.859663 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776dc02b-f1f4-4736-8350-7d8c7a1cccfa admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:01.871503 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1039/2088] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 08:58:01 2022] PUT /identity/v3/projects/e7a58a430a394ef280ccb2ca1c17a1e1/users/520fe6cbf9464583a38c6dabd0b86f8c/roles/b870cd1acaa744e1ae1d58b9ac283821 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:58:01.882758 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d4ba6b-b431-403a-9a0b-141497638689 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:01.883062 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d4ba6b-b431-403a-9a0b-141497638689 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:01.883437 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d4ba6b-b431-403a-9a0b-141497638689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:01.916343 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-51d4ba6b-b431-403a-9a0b-141497638689 None None] MFA Rules not processed for user `520fe6cbf9464583a38c6dabd0b86f8c`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:01.917638 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-51d4ba6b-b431-403a-9a0b-141497638689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:01.922088 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:02.005579 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1050/2089] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:58:01 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:58:02.018353 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776dc02b-f1f4-4736-8350-7d8c7a1cccfa admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:02.020246 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-776dc02b-f1f4-4736-8350-7d8c7a1cccfa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:02.086383 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776dc02b-f1f4-4736-8350-7d8c7a1cccfa admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:02.090353 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc52b41-be89-4253-9b30-922059af8413 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbb9ac667f04c35a63b2689df18efed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:02.090977 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc52b41-be89-4253-9b30-922059af8413 service cinder] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:02.091997 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc52b41-be89-4253-9b30-922059af8413 service cinder] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:02.091997 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc52b41-be89-4253-9b30-922059af8413 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:02.181772 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc52b41-be89-4253-9b30-922059af8413 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:02.183286 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc52b41-be89-4253-9b30-922059af8413 service cinder] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:02.193602 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1040/2090] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:58:02 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:58:06.118531 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d4ba6b-b431-403a-9a0b-141497638689 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:06.132481 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d4ba6b-b431-403a-9a0b-141497638689 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:06.136265 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec400cb-1b82-435e-8426-896c700dc1c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbb9ac667f04c35a63b2689df18efed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:06.136936 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec400cb-1b82-435e-8426-896c700dc1c6 service cinder] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:06.137225 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec400cb-1b82-435e-8426-896c700dc1c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:06.137521 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec400cb-1b82-435e-8426-896c700dc1c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:06.209799 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec400cb-1b82-435e-8426-896c700dc1c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:06.211420 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec400cb-1b82-435e-8426-896c700dc1c6 service cinder] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:06.222315 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1051/2091] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:58:06 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:58:08.157346 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc52b41-be89-4253-9b30-922059af8413 service cinder] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:08.178405 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc52b41-be89-4253-9b30-922059af8413 service cinder] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:08.182799 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4008036-cf57-4f26-bdd7-e8910b651b86 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbb9ac667f04c35a63b2689df18efed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:08.183533 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4008036-cf57-4f26-bdd7-e8910b651b86 service cinder] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:08.183917 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4008036-cf57-4f26-bdd7-e8910b651b86 service cinder] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:08.184282 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4008036-cf57-4f26-bdd7-e8910b651b86 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:08.195323 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4008036-cf57-4f26-bdd7-e8910b651b86 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:08.196807 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4008036-cf57-4f26-bdd7-e8910b651b86 service cinder] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:08.210754 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1041/2092] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:58:08 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:58:14.920931 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e375d4-a9c3-49db-869e-fd5d45b0c683 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:14.920931 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e375d4-a9c3-49db-869e-fd5d45b0c683 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:14.920931 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e375d4-a9c3-49db-869e-fd5d45b0c683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:14.944499 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-96e375d4-a9c3-49db-869e-fd5d45b0c683 None None] MFA Rules not processed for user `f652a368b1cc42ae8b6f6c7a6dff6f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:14.947066 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-96e375d4-a9c3-49db-869e-fd5d45b0c683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:15.022704 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1052/2093] 10.222.0.55 () {58 vars in 1009 bytes} [Wed Aug 24 08:58:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:58:15.082578 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4008036-cf57-4f26-bdd7-e8910b651b86 service cinder] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:15.084158 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-d4008036-cf57-4f26-bdd7-e8910b651b86 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:15.122643 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:15.170265 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4008036-cf57-4f26-bdd7-e8910b651b86 service cinder] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:15.175899 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6e6b48-8120-4023-997f-acc568572227 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0ebb5ccabef45378f0c1da69721e051', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:15.176568 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6e6b48-8120-4023-997f-acc568572227 service swift] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:15.176846 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6e6b48-8120-4023-997f-acc568572227 service swift] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:15.177136 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6e6b48-8120-4023-997f-acc568572227 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:15.186388 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6e6b48-8120-4023-997f-acc568572227 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:15.188691 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6e6b48-8120-4023-997f-acc568572227 service swift] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:15.323122 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1042/2094] 10.222.0.55 () {60 vars in 1384 bytes} [Wed Aug 24 08:58:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 08:58:25.014771 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=80 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:25.456434 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=19 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:26.893001 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e375d4-a9c3-49db-869e-fd5d45b0c683 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:27.047140 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e375d4-a9c3-49db-869e-fd5d45b0c683 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:27.060116 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86d2302-8c41-4dd6-93ed-63556a22cd7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:27.061435 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86d2302-8c41-4dd6-93ed-63556a22cd7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:27.062475 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86d2302-8c41-4dd6-93ed-63556a22cd7a admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:27.062475 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86d2302-8c41-4dd6-93ed-63556a22cd7a admin admin] PATH_INFO: `/v3/users/31050a52d1954370ba1a9be073212401` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:27.067477 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86d2302-8c41-4dd6-93ed-63556a22cd7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=31050a52d1954370ba1a9be073212401)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:27.069041 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86d2302-8c41-4dd6-93ed-63556a22cd7a admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:27.381919 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-b86d2302-8c41-4dd6-93ed-63556a22cd7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31050a52d1954370ba1a9be073212401'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:27.382700 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-b86d2302-8c41-4dd6-93ed-63556a22cd7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31050a52d1954370ba1a9be073212401'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:27.383400 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-b86d2302-8c41-4dd6-93ed-63556a22cd7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31050a52d1954370ba1a9be073212401'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:27.405930 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-b86d2302-8c41-4dd6-93ed-63556a22cd7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31050a52d1954370ba1a9be073212401'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:27.507054 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-b86d2302-8c41-4dd6-93ed-63556a22cd7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31050a52d1954370ba1a9be073212401'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:27.535962 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1053/2095] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 08:58:26 2022] DELETE /identity/v3/users/31050a52d1954370ba1a9be073212401 => generated 0 bytes in 641 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:58:27.538662 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6d5e16-0cc3-40d5-98c5-318d9a598aab None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:27.539259 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6d5e16-0cc3-40d5-98c5-318d9a598aab None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:27.539716 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6d5e16-0cc3-40d5-98c5-318d9a598aab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:27.605109 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-0e6d5e16-0cc3-40d5-98c5-318d9a598aab None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:27.608727 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-0e6d5e16-0cc3-40d5-98c5-318d9a598aab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:27.773071 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1043/2096] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:58:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:58:27.790401 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86d2302-8c41-4dd6-93ed-63556a22cd7a admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:27.792008 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-b86d2302-8c41-4dd6-93ed-63556a22cd7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:27.913800 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86d2302-8c41-4dd6-93ed-63556a22cd7a admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:27.923716 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4d980b-008a-4e95-89e5-17580a7deb86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:27.925011 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4d980b-008a-4e95-89e5-17580a7deb86 service neutron] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:27.925777 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4d980b-008a-4e95-89e5-17580a7deb86 service neutron] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:27.927356 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4d980b-008a-4e95-89e5-17580a7deb86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:27.964920 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4d980b-008a-4e95-89e5-17580a7deb86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:27.972694 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4d980b-008a-4e95-89e5-17580a7deb86 service neutron] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:28.022229 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1054/2097] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:58:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:58:28.910405 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6d5e16-0cc3-40d5-98c5-318d9a598aab None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:28.912183 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-0e6d5e16-0cc3-40d5-98c5-318d9a598aab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:28.925929 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6d5e16-0cc3-40d5-98c5-318d9a598aab None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:28.931174 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f3a7c9-f94a-4d14-908a-b3b3671198fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:28.931852 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f3a7c9-f94a-4d14-908a-b3b3671198fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:28.932136 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f3a7c9-f94a-4d14-908a-b3b3671198fb admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:28.932415 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f3a7c9-f94a-4d14-908a-b3b3671198fb admin admin] PATH_INFO: `/v3/projects/037efe7276854e73a080b845ca1c4c04` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:28.933802 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f3a7c9-f94a-4d14-908a-b3b3671198fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=037efe7276854e73a080b845ca1c4c04)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:28.935894 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f3a7c9-f94a-4d14-908a-b3b3671198fb admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:29.157665 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-37f3a7c9-f94a-4d14-908a-b3b3671198fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '037efe7276854e73a080b845ca1c4c04', 'request_id': 'req-37f3a7c9-f94a-4d14-908a-b3b3671198fb'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:29.252751 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-37f3a7c9-f94a-4d14-908a-b3b3671198fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '037efe7276854e73a080b845ca1c4c04', 'request_id': 'req-37f3a7c9-f94a-4d14-908a-b3b3671198fb'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:29.260293 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-37f3a7c9-f94a-4d14-908a-b3b3671198fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '037efe7276854e73a080b845ca1c4c04', 'request_id': 'req-37f3a7c9-f94a-4d14-908a-b3b3671198fb'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:29.273270 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-37f3a7c9-f94a-4d14-908a-b3b3671198fb admin admin] The token cache is being invalidate because project 037efe7276854e73a080b845ca1c4c04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:58:29.273581 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-37f3a7c9-f94a-4d14-908a-b3b3671198fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:29.274258 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-37f3a7c9-f94a-4d14-908a-b3b3671198fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:29.275873 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1044/2098] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 08:58:28 2022] DELETE /identity/v3/projects/037efe7276854e73a080b845ca1c4c04 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:58:29.284825 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4d980b-008a-4e95-89e5-17580a7deb86 service neutron] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:29.301859 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72a75f9-1503-4dba-b9c0-a139e012dc3b None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:29.302374 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72a75f9-1503-4dba-b9c0-a139e012dc3b None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:29.302828 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72a75f9-1503-4dba-b9c0-a139e012dc3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:29.311751 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4d980b-008a-4e95-89e5-17580a7deb86 service neutron] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:29.335672 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-b72a75f9-1503-4dba-b9c0-a139e012dc3b None None] MFA Rules not processed for user `f56524e822e641bc9bb11eddfe27f85d`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:29.337352 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-b72a75f9-1503-4dba-b9c0-a139e012dc3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:29.346919 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:29.413905 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1045/2099] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:58:29 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:58:29.425222 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889a9937-3c71-43e4-a2e0-718d4f3b5dff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:29.425222 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889a9937-3c71-43e4-a2e0-718d4f3b5dff admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:29.425222 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889a9937-3c71-43e4-a2e0-718d4f3b5dff admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:29.425222 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889a9937-3c71-43e4-a2e0-718d4f3b5dff admin admin] PATH_INFO: `/v3/users/a56c3f5859484ea28194a157570570f7` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:29.426449 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889a9937-3c71-43e4-a2e0-718d4f3b5dff admin admin] RBAC: Authorizing `identity:delete_user(user_id=a56c3f5859484ea28194a157570570f7)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:29.426449 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889a9937-3c71-43e4-a2e0-718d4f3b5dff admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:29.662812 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-889a9937-3c71-43e4-a2e0-718d4f3b5dff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a56c3f5859484ea28194a157570570f7'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:29.663431 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-889a9937-3c71-43e4-a2e0-718d4f3b5dff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a56c3f5859484ea28194a157570570f7'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:29.664186 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-889a9937-3c71-43e4-a2e0-718d4f3b5dff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a56c3f5859484ea28194a157570570f7'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:29.680351 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-889a9937-3c71-43e4-a2e0-718d4f3b5dff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a56c3f5859484ea28194a157570570f7'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:29.780815 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-889a9937-3c71-43e4-a2e0-718d4f3b5dff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a56c3f5859484ea28194a157570570f7'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:29.798653 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1055/2100] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 08:58:29 2022] DELETE /identity/v3/users/a56c3f5859484ea28194a157570570f7 => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:58:30.420898 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72a75f9-1503-4dba-b9c0-a139e012dc3b None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:30.424445 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-b72a75f9-1503-4dba-b9c0-a139e012dc3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:30.456991 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72a75f9-1503-4dba-b9c0-a139e012dc3b None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:30.544608 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:30.584352 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292b41fe-700e-4c34-86ba-df887eb0ddc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:30.586905 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292b41fe-700e-4c34-86ba-df887eb0ddc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:30.587205 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292b41fe-700e-4c34-86ba-df887eb0ddc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:30.587483 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292b41fe-700e-4c34-86ba-df887eb0ddc6 admin admin] PATH_INFO: `/v3/projects/f9d0d2e9249d47169531aaa5ee062621` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:30.589163 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292b41fe-700e-4c34-86ba-df887eb0ddc6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9d0d2e9249d47169531aaa5ee062621)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:30.591157 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292b41fe-700e-4c34-86ba-df887eb0ddc6 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:30.743732 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-292b41fe-700e-4c34-86ba-df887eb0ddc6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9d0d2e9249d47169531aaa5ee062621', 'request_id': 'req-292b41fe-700e-4c34-86ba-df887eb0ddc6'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:30.825708 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-292b41fe-700e-4c34-86ba-df887eb0ddc6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9d0d2e9249d47169531aaa5ee062621', 'request_id': 'req-292b41fe-700e-4c34-86ba-df887eb0ddc6'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:30.833377 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-292b41fe-700e-4c34-86ba-df887eb0ddc6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9d0d2e9249d47169531aaa5ee062621', 'request_id': 'req-292b41fe-700e-4c34-86ba-df887eb0ddc6'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:30.845088 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-292b41fe-700e-4c34-86ba-df887eb0ddc6 admin admin] The token cache is being invalidate because project f9d0d2e9249d47169531aaa5ee062621 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:58:30.845416 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-292b41fe-700e-4c34-86ba-df887eb0ddc6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:30.846127 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-292b41fe-700e-4c34-86ba-df887eb0ddc6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:30.848755 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1046/2101] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 08:58:30 2022] DELETE /identity/v3/projects/f9d0d2e9249d47169531aaa5ee062621 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:58:30.853428 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889a9937-3c71-43e4-a2e0-718d4f3b5dff admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:30.856223 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-889a9937-3c71-43e4-a2e0-718d4f3b5dff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:30.891620 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889a9937-3c71-43e4-a2e0-718d4f3b5dff admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:31.094295 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d014b71b-14a1-4b8a-910c-18e1573b50c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:31.095722 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d014b71b-14a1-4b8a-910c-18e1573b50c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:31.096230 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d014b71b-14a1-4b8a-910c-18e1573b50c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:31.096886 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d014b71b-14a1-4b8a-910c-18e1573b50c5 admin admin] PATH_INFO: `/v3/users/520fe6cbf9464583a38c6dabd0b86f8c` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:31.099231 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d014b71b-14a1-4b8a-910c-18e1573b50c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=520fe6cbf9464583a38c6dabd0b86f8c)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:31.103758 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d014b71b-14a1-4b8a-910c-18e1573b50c5 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:31.483699 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:31.490764 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-d014b71b-14a1-4b8a-910c-18e1573b50c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '520fe6cbf9464583a38c6dabd0b86f8c'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:31.491756 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-d014b71b-14a1-4b8a-910c-18e1573b50c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '520fe6cbf9464583a38c6dabd0b86f8c'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:31.492565 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-d014b71b-14a1-4b8a-910c-18e1573b50c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '520fe6cbf9464583a38c6dabd0b86f8c'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:31.535818 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-d014b71b-14a1-4b8a-910c-18e1573b50c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '520fe6cbf9464583a38c6dabd0b86f8c'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:31.658552 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-d014b71b-14a1-4b8a-910c-18e1573b50c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '520fe6cbf9464583a38c6dabd0b86f8c'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:31.679450 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1056/2102] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 08:58:30 2022] DELETE /identity/v3/users/520fe6cbf9464583a38c6dabd0b86f8c => generated 0 bytes in 827 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:58:32.335691 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292b41fe-700e-4c34-86ba-df887eb0ddc6 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:32.337702 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-292b41fe-700e-4c34-86ba-df887eb0ddc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:32.360771 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292b41fe-700e-4c34-86ba-df887eb0ddc6 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:32.469548 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2599973-22a5-41db-962d-bbca44b7dc03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:32.470149 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2599973-22a5-41db-962d-bbca44b7dc03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:32.470663 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2599973-22a5-41db-962d-bbca44b7dc03 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:32.471013 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2599973-22a5-41db-962d-bbca44b7dc03 admin admin] PATH_INFO: `/v3/projects/e7a58a430a394ef280ccb2ca1c17a1e1` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:32.472314 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2599973-22a5-41db-962d-bbca44b7dc03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7a58a430a394ef280ccb2ca1c17a1e1)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:32.474390 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2599973-22a5-41db-962d-bbca44b7dc03 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:32.724298 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-b2599973-22a5-41db-962d-bbca44b7dc03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7a58a430a394ef280ccb2ca1c17a1e1', 'request_id': 'req-b2599973-22a5-41db-962d-bbca44b7dc03'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:32.873052 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-b2599973-22a5-41db-962d-bbca44b7dc03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7a58a430a394ef280ccb2ca1c17a1e1', 'request_id': 'req-b2599973-22a5-41db-962d-bbca44b7dc03'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:32.909391 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-b2599973-22a5-41db-962d-bbca44b7dc03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7a58a430a394ef280ccb2ca1c17a1e1', 'request_id': 'req-b2599973-22a5-41db-962d-bbca44b7dc03'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:32.917452 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:32.937003 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-b2599973-22a5-41db-962d-bbca44b7dc03 admin admin] The token cache is being invalidate because project e7a58a430a394ef280ccb2ca1c17a1e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:58:32.938941 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-b2599973-22a5-41db-962d-bbca44b7dc03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:32.939945 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-b2599973-22a5-41db-962d-bbca44b7dc03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:32.942423 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1047/2103] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 08:58:32 2022] DELETE /identity/v3/projects/e7a58a430a394ef280ccb2ca1c17a1e1 => generated 0 bytes in 608 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:58:32.947463 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f91afa-1820-492a-8895-881d55aa84a5 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:32.952721 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f91afa-1820-492a-8895-881d55aa84a5 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:32.952721 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f91afa-1820-492a-8895-881d55aa84a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:32.984226 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-f9f91afa-1820-492a-8895-881d55aa84a5 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:32.994639 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-f9f91afa-1820-492a-8895-881d55aa84a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:33.180275 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1057/2104] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:58:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:58:33.182846 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045d819e-292b-40a7-8311-4cc026cb55bc None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:33.183966 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045d819e-292b-40a7-8311-4cc026cb55bc None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:33.184568 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045d819e-292b-40a7-8311-4cc026cb55bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:33.217600 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-045d819e-292b-40a7-8311-4cc026cb55bc None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:33.219739 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-045d819e-292b-40a7-8311-4cc026cb55bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:33.234684 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f91afa-1820-492a-8895-881d55aa84a5 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:33.240665 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1048/2105] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:58:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:58:33.247193 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f91afa-1820-492a-8895-881d55aa84a5 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:33.252734 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352a9c2c-c5c5-401a-8c99-50249f345e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:33.253391 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352a9c2c-c5c5-401a-8c99-50249f345e99 admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:33.254078 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352a9c2c-c5c5-401a-8c99-50249f345e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:33.254378 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352a9c2c-c5c5-401a-8c99-50249f345e99 admin admin] PATH_INFO: `/v3/domains` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:33.255034 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352a9c2c-c5c5-401a-8c99-50249f345e99 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:58:33.255614 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352a9c2c-c5c5-401a-8c99-50249f345e99 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:33.257359 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352a9c2c-c5c5-401a-8c99-50249f345e99 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:33.278983 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1058/2106] 10.222.0.55 () {58 vars in 1153 bytes} [Wed Aug 24 08:58:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:58:33.279890 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045d819e-292b-40a7-8311-4cc026cb55bc None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:33.297500 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045d819e-292b-40a7-8311-4cc026cb55bc None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:33.303538 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e75485d-55f4-49c6-978f-952a96ef3d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:33.304287 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e75485d-55f4-49c6-978f-952a96ef3d02 admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:33.304634 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e75485d-55f4-49c6-978f-952a96ef3d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:33.304970 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e75485d-55f4-49c6-978f-952a96ef3d02 admin admin] PATH_INFO: `/v3/projects` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:33.305789 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e75485d-55f4-49c6-978f-952a96ef3d02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:33.308334 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e75485d-55f4-49c6-978f-952a96ef3d02 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:33.417421 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1049/2107] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 08:58:33 2022] POST /identity/v3/projects => generated 385 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:58:33.421363 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352a9c2c-c5c5-401a-8c99-50249f345e99 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:33.435113 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352a9c2c-c5c5-401a-8c99-50249f345e99 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:33.543395 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd09e80-2ff4-486e-8789-3e184e8383d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:33.545494 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd09e80-2ff4-486e-8789-3e184e8383d5 admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:33.545494 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd09e80-2ff4-486e-8789-3e184e8383d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:33.545494 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd09e80-2ff4-486e-8789-3e184e8383d5 admin admin] PATH_INFO: `/v3/users` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:33.545742 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd09e80-2ff4-486e-8789-3e184e8383d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:33.550348 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd09e80-2ff4-486e-8789-3e184e8383d5 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:33.651100 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1059/2108] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 08:58:33 2022] POST /identity/v3/users => generated 293 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:58:33.657365 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e75485d-55f4-49c6-978f-952a96ef3d02 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:33.683291 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e75485d-55f4-49c6-978f-952a96ef3d02 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:33.691922 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e991799c-91c2-47a6-905a-60a15fec233c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:33.693087 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e991799c-91c2-47a6-905a-60a15fec233c admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:33.693582 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e991799c-91c2-47a6-905a-60a15fec233c admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:33.697681 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e991799c-91c2-47a6-905a-60a15fec233c admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:33.697681 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e991799c-91c2-47a6-905a-60a15fec233c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:33.698266 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e991799c-91c2-47a6-905a-60a15fec233c admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:33.732309 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1050/2109] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:58:33 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:58:33.736589 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd09e80-2ff4-486e-8789-3e184e8383d5 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:33.747989 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd09e80-2ff4-486e-8789-3e184e8383d5 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:33.768410 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4617f2-ad23-4bda-a3e1-747fd101092c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:33.769109 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4617f2-ad23-4bda-a3e1-747fd101092c admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:33.769389 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4617f2-ad23-4bda-a3e1-747fd101092c admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:33.769661 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4617f2-ad23-4bda-a3e1-747fd101092c admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:33.770333 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4617f2-ad23-4bda-a3e1-747fd101092c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:33.771989 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4617f2-ad23-4bda-a3e1-747fd101092c admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:33.810296 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1060/2110] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:58:33 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:58:33.816567 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e991799c-91c2-47a6-905a-60a15fec233c admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:33.836883 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e991799c-91c2-47a6-905a-60a15fec233c admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:33.847015 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d4899d-ebcb-442d-95ce-b7fc13aba59b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:33.848861 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d4899d-ebcb-442d-95ce-b7fc13aba59b admin admin] REQUEST_METHOD: `PUT` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:33.851871 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d4899d-ebcb-442d-95ce-b7fc13aba59b admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:33.851871 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d4899d-ebcb-442d-95ce-b7fc13aba59b admin admin] PATH_INFO: `/v3/projects/9137fdb1ccc3433cad5c223e0df83d21/users/9165197817204690bee2229b9d14ad56/roles/b674fd0a592e4e0cada811bba5ceea4d` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:33.880524 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d4899d-ebcb-442d-95ce-b7fc13aba59b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9137fdb1ccc3433cad5c223e0df83d21, user_id=9165197817204690bee2229b9d14ad56, role_id=b674fd0a592e4e0cada811bba5ceea4d)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:33.885723 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d4899d-ebcb-442d-95ce-b7fc13aba59b admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:34.036395 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1051/2111] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 08:58:33 2022] PUT /identity/v3/projects/9137fdb1ccc3433cad5c223e0df83d21/users/9165197817204690bee2229b9d14ad56/roles/b674fd0a592e4e0cada811bba5ceea4d => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:58:34.054801 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6254e866-046c-4836-8d02-4d84aca550cc None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:34.055652 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6254e866-046c-4836-8d02-4d84aca550cc None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:34.055827 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6254e866-046c-4836-8d02-4d84aca550cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:34.104161 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-6254e866-046c-4836-8d02-4d84aca550cc None None] MFA Rules not processed for user `9165197817204690bee2229b9d14ad56`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:34.106400 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-6254e866-046c-4836-8d02-4d84aca550cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:34.137067 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:34.283872 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d4899d-ebcb-442d-95ce-b7fc13aba59b admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:34.303906 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1061/2112] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:58:34 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:58:34.451596 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d4899d-ebcb-442d-95ce-b7fc13aba59b admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:34.460522 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f93ce0-5a1d-4c93-9692-188e8f848002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:34.461456 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f93ce0-5a1d-4c93-9692-188e8f848002 admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:34.461847 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f93ce0-5a1d-4c93-9692-188e8f848002 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:34.462223 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f93ce0-5a1d-4c93-9692-188e8f848002 admin admin] PATH_INFO: `/v3/projects` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:34.463088 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f93ce0-5a1d-4c93-9692-188e8f848002 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:34.465442 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f93ce0-5a1d-4c93-9692-188e8f848002 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:34.633266 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1052/2113] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 08:58:34 2022] POST /identity/v3/projects => generated 385 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:58:34.639111 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6254e866-046c-4836-8d02-4d84aca550cc None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:34.657077 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6254e866-046c-4836-8d02-4d84aca550cc None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:34.754845 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1934e09d-349d-448b-b14a-73c923fdad7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:34.755891 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1934e09d-349d-448b-b14a-73c923fdad7f admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:34.756449 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1934e09d-349d-448b-b14a-73c923fdad7f admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:34.760618 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1934e09d-349d-448b-b14a-73c923fdad7f admin admin] PATH_INFO: `/v3/users` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:34.760618 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1934e09d-349d-448b-b14a-73c923fdad7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:34.762860 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1934e09d-349d-448b-b14a-73c923fdad7f admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:34.856387 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1062/2114] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 08:58:34 2022] POST /identity/v3/users => generated 293 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:58:34.862945 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f93ce0-5a1d-4c93-9692-188e8f848002 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:34.875389 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f93ce0-5a1d-4c93-9692-188e8f848002 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:34.880828 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d6490a-6804-4fda-ae70-3d2c171e7afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:34.881509 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d6490a-6804-4fda-ae70-3d2c171e7afc admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:34.881795 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d6490a-6804-4fda-ae70-3d2c171e7afc admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:34.882086 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d6490a-6804-4fda-ae70-3d2c171e7afc admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:34.882906 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d6490a-6804-4fda-ae70-3d2c171e7afc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:34.884681 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d6490a-6804-4fda-ae70-3d2c171e7afc admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:34.897059 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1053/2115] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:58:34 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:58:34.901264 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1934e09d-349d-448b-b14a-73c923fdad7f admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:34.914588 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1934e09d-349d-448b-b14a-73c923fdad7f admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:34.919913 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91567d7-fda8-42aa-8304-bc19206c313f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:34.925288 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91567d7-fda8-42aa-8304-bc19206c313f admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:34.925288 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91567d7-fda8-42aa-8304-bc19206c313f admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:34.925288 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91567d7-fda8-42aa-8304-bc19206c313f admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:34.925288 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91567d7-fda8-42aa-8304-bc19206c313f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:34.925288 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91567d7-fda8-42aa-8304-bc19206c313f admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:34.936014 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1063/2116] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:58:34 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:58:34.941822 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d6490a-6804-4fda-ae70-3d2c171e7afc admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:34.951357 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d6490a-6804-4fda-ae70-3d2c171e7afc admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:34.957080 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71065272-c022-4599-b298-980933c76577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:34.957743 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71065272-c022-4599-b298-980933c76577 admin admin] REQUEST_METHOD: `PUT` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:34.958014 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71065272-c022-4599-b298-980933c76577 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:34.958284 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71065272-c022-4599-b298-980933c76577 admin admin] PATH_INFO: `/v3/projects/1f961c8e09ee48e191b03b6db5b5302b/users/c8385eb0e5eb40b09c384b41c4af5f83/roles/b674fd0a592e4e0cada811bba5ceea4d` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:34.974189 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71065272-c022-4599-b298-980933c76577 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f961c8e09ee48e191b03b6db5b5302b, user_id=c8385eb0e5eb40b09c384b41c4af5f83, role_id=b674fd0a592e4e0cada811bba5ceea4d)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:34.976875 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71065272-c022-4599-b298-980933c76577 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:35.079867 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1054/2117] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 08:58:34 2022] PUT /identity/v3/projects/1f961c8e09ee48e191b03b6db5b5302b/users/c8385eb0e5eb40b09c384b41c4af5f83/roles/b674fd0a592e4e0cada811bba5ceea4d => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:58:35.091265 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f1a01-9e23-4254-8ec4-681705ad2d6e None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:35.091987 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f1a01-9e23-4254-8ec4-681705ad2d6e None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:35.092570 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f1a01-9e23-4254-8ec4-681705ad2d6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:35.144494 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-b90f1a01-9e23-4254-8ec4-681705ad2d6e None None] MFA Rules not processed for user `c8385eb0e5eb40b09c384b41c4af5f83`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:35.145707 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-b90f1a01-9e23-4254-8ec4-681705ad2d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:35.218039 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1064/2118] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:58:35 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:58:35.224525 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71065272-c022-4599-b298-980933c76577 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:35.269414 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:35.301804 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71065272-c022-4599-b298-980933c76577 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:35.307516 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e014cbaf-5f41-4586-8276-161b095c18ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:35.310436 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e014cbaf-5f41-4586-8276-161b095c18ed admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:35.310436 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e014cbaf-5f41-4586-8276-161b095c18ed admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:35.310436 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e014cbaf-5f41-4586-8276-161b095c18ed admin admin] PATH_INFO: `/v3/projects` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:35.310436 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e014cbaf-5f41-4586-8276-161b095c18ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:35.311525 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e014cbaf-5f41-4586-8276-161b095c18ed admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:35.455159 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1055/2119] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 08:58:35 2022] POST /identity/v3/projects => generated 385 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:58:35.459841 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90f1a01-9e23-4254-8ec4-681705ad2d6e None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:35.471828 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90f1a01-9e23-4254-8ec4-681705ad2d6e None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:35.495961 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:35.551449 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60598b95-7ad0-4f46-a450-560ba7cf36fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:35.553034 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60598b95-7ad0-4f46-a450-560ba7cf36fc admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:35.553034 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60598b95-7ad0-4f46-a450-560ba7cf36fc admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:35.553330 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60598b95-7ad0-4f46-a450-560ba7cf36fc admin admin] PATH_INFO: `/v3/users` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:35.553758 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60598b95-7ad0-4f46-a450-560ba7cf36fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:35.556270 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60598b95-7ad0-4f46-a450-560ba7cf36fc admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:35.659566 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1065/2120] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 08:58:35 2022] POST /identity/v3/users => generated 299 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:58:35.664172 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e014cbaf-5f41-4586-8276-161b095c18ed admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:35.675635 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e014cbaf-5f41-4586-8276-161b095c18ed admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:35.680860 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5561f826-d981-4b68-b160-ba41df56d5f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:35.681652 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5561f826-d981-4b68-b160-ba41df56d5f3 admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:35.681959 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5561f826-d981-4b68-b160-ba41df56d5f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:35.682228 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5561f826-d981-4b68-b160-ba41df56d5f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:35.682892 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5561f826-d981-4b68-b160-ba41df56d5f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:35.684726 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5561f826-d981-4b68-b160-ba41df56d5f3 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:35.697552 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1056/2121] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:58:35 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 08:58:35.702509 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60598b95-7ad0-4f46-a450-560ba7cf36fc admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:35.714309 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60598b95-7ad0-4f46-a450-560ba7cf36fc admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:35.719445 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b941cb-5a26-42d3-a3ef-66d04a4cb8c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:35.720178 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b941cb-5a26-42d3-a3ef-66d04a4cb8c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:35.720495 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b941cb-5a26-42d3-a3ef-66d04a4cb8c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:35.720961 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b941cb-5a26-42d3-a3ef-66d04a4cb8c6 admin admin] PATH_INFO: `/v3/projects/c0bf4f682a334b679f3d0ce761bcbbc8/users/43507f98d0c4407e87d00cdbb8c7cd7f/roles/b870cd1acaa744e1ae1d58b9ac283821` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:35.739543 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b941cb-5a26-42d3-a3ef-66d04a4cb8c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0bf4f682a334b679f3d0ce761bcbbc8, user_id=43507f98d0c4407e87d00cdbb8c7cd7f, role_id=b870cd1acaa744e1ae1d58b9ac283821)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:35.742429 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b941cb-5a26-42d3-a3ef-66d04a4cb8c6 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:35.841097 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1066/2122] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 08:58:35 2022] PUT /identity/v3/projects/c0bf4f682a334b679f3d0ce761bcbbc8/users/43507f98d0c4407e87d00cdbb8c7cd7f/roles/b870cd1acaa744e1ae1d58b9ac283821 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:58:35.845696 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5561f826-d981-4b68-b160-ba41df56d5f3 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:35.856932 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5561f826-d981-4b68-b160-ba41df56d5f3 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:35.926849 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af62ea6b-dbc2-4be6-916f-da80b88fc340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:35.927543 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af62ea6b-dbc2-4be6-916f-da80b88fc340 admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:35.927937 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af62ea6b-dbc2-4be6-916f-da80b88fc340 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:35.928254 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af62ea6b-dbc2-4be6-916f-da80b88fc340 admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:35.929103 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af62ea6b-dbc2-4be6-916f-da80b88fc340 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:35.931051 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af62ea6b-dbc2-4be6-916f-da80b88fc340 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:35.943512 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1057/2123] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:58:35 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:58:35.948268 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b941cb-5a26-42d3-a3ef-66d04a4cb8c6 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:35.962464 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b941cb-5a26-42d3-a3ef-66d04a4cb8c6 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:35.968270 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c32305-7a0f-434a-a175-a6ba2be11c29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:35.969466 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c32305-7a0f-434a-a175-a6ba2be11c29 admin admin] REQUEST_METHOD: `PUT` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:35.969749 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c32305-7a0f-434a-a175-a6ba2be11c29 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:35.970232 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c32305-7a0f-434a-a175-a6ba2be11c29 admin admin] PATH_INFO: `/v3/projects/c0bf4f682a334b679f3d0ce761bcbbc8/users/43507f98d0c4407e87d00cdbb8c7cd7f/roles/b870cd1acaa744e1ae1d58b9ac283821` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:35.972372 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c32305-7a0f-434a-a175-a6ba2be11c29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0bf4f682a334b679f3d0ce761bcbbc8, user_id=43507f98d0c4407e87d00cdbb8c7cd7f, role_id=b870cd1acaa744e1ae1d58b9ac283821)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:35.975179 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c32305-7a0f-434a-a175-a6ba2be11c29 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:35.987520 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1067/2124] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 08:58:35 2022] PUT /identity/v3/projects/c0bf4f682a334b679f3d0ce761bcbbc8/users/43507f98d0c4407e87d00cdbb8c7cd7f/roles/b870cd1acaa744e1ae1d58b9ac283821 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:58:35.999302 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0c304e-55a7-4ec3-9fa6-901efcdd06ec None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:35.999586 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0c304e-55a7-4ec3-9fa6-901efcdd06ec None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:35.999918 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0c304e-55a7-4ec3-9fa6-901efcdd06ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:36.030705 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-ff0c304e-55a7-4ec3-9fa6-901efcdd06ec None None] MFA Rules not processed for user `43507f98d0c4407e87d00cdbb8c7cd7f`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:36.033002 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-ff0c304e-55a7-4ec3-9fa6-901efcdd06ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:36.126220 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1058/2125] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:58:35 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:58:36.144070 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c32305-7a0f-434a-a175-a6ba2be11c29 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:36.147527 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-31c32305-7a0f-434a-a175-a6ba2be11c29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:36.264877 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c32305-7a0f-434a-a175-a6ba2be11c29 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:36.269824 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2b8286-9abc-4b27-8af0-7c8f0218bb56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbb9ac667f04c35a63b2689df18efed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:36.270862 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2b8286-9abc-4b27-8af0-7c8f0218bb56 service cinder] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:36.271171 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2b8286-9abc-4b27-8af0-7c8f0218bb56 service cinder] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:36.271520 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2b8286-9abc-4b27-8af0-7c8f0218bb56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:36.376670 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2b8286-9abc-4b27-8af0-7c8f0218bb56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:36.378143 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2b8286-9abc-4b27-8af0-7c8f0218bb56 service cinder] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:36.399307 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1068/2126] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:58:36 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:58:39.783656 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d59d02-dd8d-4631-83bf-db335a01f462 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:39.784411 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d59d02-dd8d-4631-83bf-db335a01f462 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:39.784411 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d59d02-dd8d-4631-83bf-db335a01f462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:39.806460 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-e0d59d02-dd8d-4631-83bf-db335a01f462 None None] MFA Rules not processed for user `f652a368b1cc42ae8b6f6c7a6dff6f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:39.808206 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-e0d59d02-dd8d-4631-83bf-db335a01f462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:39.843406 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:39.896929 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1059/2127] 10.222.0.55 () {58 vars in 967 bytes} [Wed Aug 24 08:58:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:58:39.924938 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2b8286-9abc-4b27-8af0-7c8f0218bb56 service cinder] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:39.926885 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-3c2b8286-9abc-4b27-8af0-7c8f0218bb56 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:39.990455 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:40.051106 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2b8286-9abc-4b27-8af0-7c8f0218bb56 service cinder] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:40.061128 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056a4d28-4f3d-4625-a7bb-42fc147d3ac9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0ebb5ccabef45378f0c1da69721e051', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:40.062713 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056a4d28-4f3d-4625-a7bb-42fc147d3ac9 service swift] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:40.063178 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056a4d28-4f3d-4625-a7bb-42fc147d3ac9 service swift] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:40.063682 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056a4d28-4f3d-4625-a7bb-42fc147d3ac9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:40.077488 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056a4d28-4f3d-4625-a7bb-42fc147d3ac9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:40.080384 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056a4d28-4f3d-4625-a7bb-42fc147d3ac9 service swift] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:40.091388 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1069/2128] 10.222.0.55 () {60 vars in 1384 bytes} [Wed Aug 24 08:58:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 08:58:40.811474 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d59d02-dd8d-4631-83bf-db335a01f462 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:40.813316 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-e0d59d02-dd8d-4631-83bf-db335a01f462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:40.831062 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d59d02-dd8d-4631-83bf-db335a01f462 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:40.835904 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172a6393-2983-40ea-a76b-ff24f1f8ce97 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbb9ac667f04c35a63b2689df18efed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:40.836825 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172a6393-2983-40ea-a76b-ff24f1f8ce97 service cinder] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:40.837154 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172a6393-2983-40ea-a76b-ff24f1f8ce97 service cinder] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:40.838304 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172a6393-2983-40ea-a76b-ff24f1f8ce97 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:40.961067 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172a6393-2983-40ea-a76b-ff24f1f8ce97 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:40.964867 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172a6393-2983-40ea-a76b-ff24f1f8ce97 service cinder] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:41.014517 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1060/2129] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:58:40 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:58:41.448091 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f67b7a-c821-432f-b7fb-c3985150987e None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:41.450325 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f67b7a-c821-432f-b7fb-c3985150987e None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:41.453669 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f67b7a-c821-432f-b7fb-c3985150987e None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:41.462257 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1070/2130] 10.222.0.55 () {54 vars in 921 bytes} [Wed Aug 24 08:58:41 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:58:41.475012 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea38f4dc-441d-4ccf-b818-3b32cf6460a1 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:41.475978 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea38f4dc-441d-4ccf-b818-3b32cf6460a1 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:41.476785 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea38f4dc-441d-4ccf-b818-3b32cf6460a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:41.513231 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-ea38f4dc-441d-4ccf-b818-3b32cf6460a1 None None] MFA Rules not processed for user `cf0c99771661413aaa0f237af6b394ab`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:41.515473 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-ea38f4dc-441d-4ccf-b818-3b32cf6460a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:41.665999 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1061/2131] 10.222.0.55 () {58 vars in 1018 bytes} [Wed Aug 24 08:58:41 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:58:41.680312 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f67b7a-c821-432f-b7fb-c3985150987e None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:41.690258 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-75f67b7a-c821-432f-b7fb-c3985150987e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:41.713918 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f67b7a-c821-432f-b7fb-c3985150987e None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:41.723899 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c95070a-e8f1-42fb-88ca-337d7aa10d02 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf0c99771661413aaa0f237af6b394ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:41.725038 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c95070a-e8f1-42fb-88ca-337d7aa10d02 service placement] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:41.725528 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c95070a-e8f1-42fb-88ca-337d7aa10d02 service placement] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:41.725997 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c95070a-e8f1-42fb-88ca-337d7aa10d02 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:41.743685 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c95070a-e8f1-42fb-88ca-337d7aa10d02 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:41.746167 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c95070a-e8f1-42fb-88ca-337d7aa10d02 service placement] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:41.769462 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1071/2132] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:58:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:58:43.244079 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea38f4dc-441d-4ccf-b818-3b32cf6460a1 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:43.245742 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-ea38f4dc-441d-4ccf-b818-3b32cf6460a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:43.337618 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea38f4dc-441d-4ccf-b818-3b32cf6460a1 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:43.342530 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fde378c-675f-4957-8599-dd9cd4051476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:43.343194 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fde378c-675f-4957-8599-dd9cd4051476 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:43.343584 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fde378c-675f-4957-8599-dd9cd4051476 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:43.343743 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fde378c-675f-4957-8599-dd9cd4051476 admin admin] PATH_INFO: `/v3/users/70bb349cf6cc44f385c6434dfaa1b7c6` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:43.344961 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fde378c-675f-4957-8599-dd9cd4051476 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70bb349cf6cc44f385c6434dfaa1b7c6)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:43.346865 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fde378c-675f-4957-8599-dd9cd4051476 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:43.368471 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:43.403177 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c95070a-e8f1-42fb-88ca-337d7aa10d02 service placement] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:43.414276 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c95070a-e8f1-42fb-88ca-337d7aa10d02 service placement] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:43.418562 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac87a810-54f6-4fb0-8a6e-b70588870de7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbb9ac667f04c35a63b2689df18efed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:43.419252 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac87a810-54f6-4fb0-8a6e-b70588870de7 service cinder] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:43.419532 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac87a810-54f6-4fb0-8a6e-b70588870de7 service cinder] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:43.419805 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac87a810-54f6-4fb0-8a6e-b70588870de7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:43.431853 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac87a810-54f6-4fb0-8a6e-b70588870de7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:43.433440 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac87a810-54f6-4fb0-8a6e-b70588870de7 service cinder] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:43.445319 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1072/2133] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:58:43 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:58:43.600965 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-6fde378c-675f-4957-8599-dd9cd4051476 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70bb349cf6cc44f385c6434dfaa1b7c6'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:43.602076 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-6fde378c-675f-4957-8599-dd9cd4051476 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70bb349cf6cc44f385c6434dfaa1b7c6'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:43.602700 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-6fde378c-675f-4957-8599-dd9cd4051476 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70bb349cf6cc44f385c6434dfaa1b7c6'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:43.616872 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-6fde378c-675f-4957-8599-dd9cd4051476 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70bb349cf6cc44f385c6434dfaa1b7c6'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:43.744229 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-6fde378c-675f-4957-8599-dd9cd4051476 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70bb349cf6cc44f385c6434dfaa1b7c6'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:43.777151 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1062/2134] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 08:58:43 2022] DELETE /identity/v3/users/70bb349cf6cc44f385c6434dfaa1b7c6 => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:58:44.257629 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac87a810-54f6-4fb0-8a6e-b70588870de7 service cinder] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:44.276295 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac87a810-54f6-4fb0-8a6e-b70588870de7 service cinder] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:44.363075 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2d5af4-f16b-4711-ab41-ba5ec2e93cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:44.363701 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2d5af4-f16b-4711-ab41-ba5ec2e93cc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:44.363977 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2d5af4-f16b-4711-ab41-ba5ec2e93cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:44.364254 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2d5af4-f16b-4711-ab41-ba5ec2e93cc5 admin admin] PATH_INFO: `/v3/projects/bd3ebf9d2a02484c9ea5f1fe959a4618` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:44.365529 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2d5af4-f16b-4711-ab41-ba5ec2e93cc5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd3ebf9d2a02484c9ea5f1fe959a4618)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:44.367521 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2d5af4-f16b-4711-ab41-ba5ec2e93cc5 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:44.526694 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-3b2d5af4-f16b-4711-ab41-ba5ec2e93cc5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd3ebf9d2a02484c9ea5f1fe959a4618', 'request_id': 'req-3b2d5af4-f16b-4711-ab41-ba5ec2e93cc5'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:44.533366 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-3b2d5af4-f16b-4711-ab41-ba5ec2e93cc5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd3ebf9d2a02484c9ea5f1fe959a4618', 'request_id': 'req-3b2d5af4-f16b-4711-ab41-ba5ec2e93cc5'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:44.551599 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fde378c-675f-4957-8599-dd9cd4051476 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:44.554327 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-6fde378c-675f-4957-8599-dd9cd4051476 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:44.577253 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fde378c-675f-4957-8599-dd9cd4051476 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:44.643361 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a97b14-0a16-4dcc-be78-492b684da01a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:44.644005 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a97b14-0a16-4dcc-be78-492b684da01a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:44.644282 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a97b14-0a16-4dcc-be78-492b684da01a admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:44.644580 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a97b14-0a16-4dcc-be78-492b684da01a admin admin] PATH_INFO: `/v3/users/bb815fa9f45a4b1db1f30fbfaeb3c4c9` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:44.645753 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a97b14-0a16-4dcc-be78-492b684da01a admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb815fa9f45a4b1db1f30fbfaeb3c4c9)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:44.647697 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a97b14-0a16-4dcc-be78-492b684da01a admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:44.651098 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-3b2d5af4-f16b-4711-ab41-ba5ec2e93cc5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd3ebf9d2a02484c9ea5f1fe959a4618', 'request_id': 'req-3b2d5af4-f16b-4711-ab41-ba5ec2e93cc5'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:44.665456 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-3b2d5af4-f16b-4711-ab41-ba5ec2e93cc5 admin admin] The token cache is being invalidate because project bd3ebf9d2a02484c9ea5f1fe959a4618 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:58:44.665748 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-3b2d5af4-f16b-4711-ab41-ba5ec2e93cc5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:44.666401 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-3b2d5af4-f16b-4711-ab41-ba5ec2e93cc5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:44.670573 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1073/2135] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 08:58:44 2022] DELETE /identity/v3/projects/bd3ebf9d2a02484c9ea5f1fe959a4618 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:58:44.673422 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2d5af4-f16b-4711-ab41-ba5ec2e93cc5 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:44.675243 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-3b2d5af4-f16b-4711-ab41-ba5ec2e93cc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:44.691689 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2d5af4-f16b-4711-ab41-ba5ec2e93cc5 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:44.696512 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4878bc3e-8eb8-4c93-b3c0-e676d0bb8704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:44.697212 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4878bc3e-8eb8-4c93-b3c0-e676d0bb8704 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:44.697540 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4878bc3e-8eb8-4c93-b3c0-e676d0bb8704 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:44.697740 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4878bc3e-8eb8-4c93-b3c0-e676d0bb8704 admin admin] PATH_INFO: `/v3/users/de8e2b95aa4a42ef8d90cdf5787b67e8` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:44.699031 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4878bc3e-8eb8-4c93-b3c0-e676d0bb8704 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de8e2b95aa4a42ef8d90cdf5787b67e8)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:44.701219 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4878bc3e-8eb8-4c93-b3c0-e676d0bb8704 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:44.725301 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:45.019865 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-04a97b14-0a16-4dcc-be78-492b684da01a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb815fa9f45a4b1db1f30fbfaeb3c4c9'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:45.021017 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-04a97b14-0a16-4dcc-be78-492b684da01a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb815fa9f45a4b1db1f30fbfaeb3c4c9'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:45.022313 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-04a97b14-0a16-4dcc-be78-492b684da01a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb815fa9f45a4b1db1f30fbfaeb3c4c9'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:45.031535 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-4878bc3e-8eb8-4c93-b3c0-e676d0bb8704 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de8e2b95aa4a42ef8d90cdf5787b67e8'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:45.032751 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-4878bc3e-8eb8-4c93-b3c0-e676d0bb8704 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de8e2b95aa4a42ef8d90cdf5787b67e8'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:45.033641 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-4878bc3e-8eb8-4c93-b3c0-e676d0bb8704 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de8e2b95aa4a42ef8d90cdf5787b67e8'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:45.047425 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-04a97b14-0a16-4dcc-be78-492b684da01a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb815fa9f45a4b1db1f30fbfaeb3c4c9'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:45.053633 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-4878bc3e-8eb8-4c93-b3c0-e676d0bb8704 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de8e2b95aa4a42ef8d90cdf5787b67e8'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:45.127135 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-04a97b14-0a16-4dcc-be78-492b684da01a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb815fa9f45a4b1db1f30fbfaeb3c4c9'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:45.151833 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1063/2136] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 08:58:44 2022] DELETE /identity/v3/users/bb815fa9f45a4b1db1f30fbfaeb3c4c9 => generated 0 bytes in 601 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:58:45.194701 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-4878bc3e-8eb8-4c93-b3c0-e676d0bb8704 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de8e2b95aa4a42ef8d90cdf5787b67e8'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:45.221193 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1074/2137] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 08:58:44 2022] DELETE /identity/v3/users/de8e2b95aa4a42ef8d90cdf5787b67e8 => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:58:45.526432 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a97b14-0a16-4dcc-be78-492b684da01a admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:45.528864 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-04a97b14-0a16-4dcc-be78-492b684da01a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:45.566738 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:45.629996 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4878bc3e-8eb8-4c93-b3c0-e676d0bb8704 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:45.637905 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-4878bc3e-8eb8-4c93-b3c0-e676d0bb8704 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:45.770848 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a97b14-0a16-4dcc-be78-492b684da01a admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:45.780019 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941d0c51-decd-4f3a-b1bb-5e6b8868b621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:45.781052 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941d0c51-decd-4f3a-b1bb-5e6b8868b621 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:45.781517 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941d0c51-decd-4f3a-b1bb-5e6b8868b621 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:45.781931 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941d0c51-decd-4f3a-b1bb-5e6b8868b621 admin admin] PATH_INFO: `/v3/projects/f9753d09aeea45378d637bf28536bb7d` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:45.783719 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941d0c51-decd-4f3a-b1bb-5e6b8868b621 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9753d09aeea45378d637bf28536bb7d)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:45.784808 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4878bc3e-8eb8-4c93-b3c0-e676d0bb8704 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:45.787208 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941d0c51-decd-4f3a-b1bb-5e6b8868b621 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:45.790990 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490d0ad8-c28e-4652-afab-018c3ab65df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:45.791648 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490d0ad8-c28e-4652-afab-018c3ab65df9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:45.791928 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490d0ad8-c28e-4652-afab-018c3ab65df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:45.792770 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490d0ad8-c28e-4652-afab-018c3ab65df9 admin admin] PATH_INFO: `/v3/projects/182cfd3e5c574fc090dabe4c67584751` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:45.793667 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490d0ad8-c28e-4652-afab-018c3ab65df9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=182cfd3e5c574fc090dabe4c67584751)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:45.795786 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490d0ad8-c28e-4652-afab-018c3ab65df9 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:45.998997 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-490d0ad8-c28e-4652-afab-018c3ab65df9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '182cfd3e5c574fc090dabe4c67584751', 'request_id': 'req-490d0ad8-c28e-4652-afab-018c3ab65df9'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:46.003443 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-941d0c51-decd-4f3a-b1bb-5e6b8868b621 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9753d09aeea45378d637bf28536bb7d', 'request_id': 'req-941d0c51-decd-4f3a-b1bb-5e6b8868b621'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:46.006999 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-490d0ad8-c28e-4652-afab-018c3ab65df9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '182cfd3e5c574fc090dabe4c67584751', 'request_id': 'req-490d0ad8-c28e-4652-afab-018c3ab65df9'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:46.100236 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-941d0c51-decd-4f3a-b1bb-5e6b8868b621 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9753d09aeea45378d637bf28536bb7d', 'request_id': 'req-941d0c51-decd-4f3a-b1bb-5e6b8868b621'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:46.107874 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-941d0c51-decd-4f3a-b1bb-5e6b8868b621 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9753d09aeea45378d637bf28536bb7d', 'request_id': 'req-941d0c51-decd-4f3a-b1bb-5e6b8868b621'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:46.120326 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-941d0c51-decd-4f3a-b1bb-5e6b8868b621 admin admin] The token cache is being invalidate because project f9753d09aeea45378d637bf28536bb7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:58:46.120638 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-941d0c51-decd-4f3a-b1bb-5e6b8868b621 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:46.121320 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-941d0c51-decd-4f3a-b1bb-5e6b8868b621 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:46.122863 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1064/2138] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 08:58:45 2022] DELETE /identity/v3/projects/f9753d09aeea45378d637bf28536bb7d => generated 0 bytes in 598 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:58:46.128356 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f497608-508d-4941-b309-51978ab9d956 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:46.128683 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f497608-508d-4941-b309-51978ab9d956 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:46.128955 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f497608-508d-4941-b309-51978ab9d956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:46.150194 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-0f497608-508d-4941-b309-51978ab9d956 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:46.151508 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-0f497608-508d-4941-b309-51978ab9d956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:46.205505 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-490d0ad8-c28e-4652-afab-018c3ab65df9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '182cfd3e5c574fc090dabe4c67584751', 'request_id': 'req-490d0ad8-c28e-4652-afab-018c3ab65df9'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:46.221160 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1065/2139] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:58:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:58:46.221705 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-490d0ad8-c28e-4652-afab-018c3ab65df9 admin admin] The token cache is being invalidate because project 182cfd3e5c574fc090dabe4c67584751 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:58:46.222336 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-490d0ad8-c28e-4652-afab-018c3ab65df9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:46.223111 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-490d0ad8-c28e-4652-afab-018c3ab65df9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:46.225186 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1075/2140] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 08:58:45 2022] DELETE /identity/v3/projects/182cfd3e5c574fc090dabe4c67584751 => generated 0 bytes in 596 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:58:46.230243 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53fbc98-55c1-43f1-aadc-4fb40682bddd None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:46.230610 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53fbc98-55c1-43f1-aadc-4fb40682bddd None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:46.230884 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53fbc98-55c1-43f1-aadc-4fb40682bddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:46.233274 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c74656a-84a4-467b-9e3b-6cb4a4003f0c None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:46.234726 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c74656a-84a4-467b-9e3b-6cb4a4003f0c None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:46.235487 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c74656a-84a4-467b-9e3b-6cb4a4003f0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:46.256455 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-c53fbc98-55c1-43f1-aadc-4fb40682bddd None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:46.257954 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-c53fbc98-55c1-43f1-aadc-4fb40682bddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:46.268914 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1066/2141] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:58:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:58:46.276300 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-5c74656a-84a4-467b-9e3b-6cb4a4003f0c None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:46.277940 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-5c74656a-84a4-467b-9e3b-6cb4a4003f0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:46.279021 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53fbc98-55c1-43f1-aadc-4fb40682bddd None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:46.291852 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1076/2142] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:58:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:58:46.294933 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53fbc98-55c1-43f1-aadc-4fb40682bddd None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:46.302021 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7fcb81-3168-49e4-8253-385ab400f91d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:46.302613 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d114764-00cf-4f66-8e67-92f214cfb486 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:46.303028 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7fcb81-3168-49e4-8253-385ab400f91d admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:46.303463 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7fcb81-3168-49e4-8253-385ab400f91d admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:46.304064 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d114764-00cf-4f66-8e67-92f214cfb486 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:46.304182 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7fcb81-3168-49e4-8253-385ab400f91d admin admin] PATH_INFO: `/v3/domains` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:46.304947 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d114764-00cf-4f66-8e67-92f214cfb486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:46.305268 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7fcb81-3168-49e4-8253-385ab400f91d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:58:46.305815 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7fcb81-3168-49e4-8253-385ab400f91d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:46.318963 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7fcb81-3168-49e4-8253-385ab400f91d admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:46.333214 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1067/2143] 10.222.0.55 () {58 vars in 1153 bytes} [Wed Aug 24 08:58:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:58:46.336962 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-8d114764-00cf-4f66-8e67-92f214cfb486 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:46.338376 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7fcb81-3168-49e4-8253-385ab400f91d admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:46.338556 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-8d114764-00cf-4f66-8e67-92f214cfb486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:46.347909 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1077/2144] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:58:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:58:46.349809 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7fcb81-3168-49e4-8253-385ab400f91d admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:46.355162 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d715ec-751b-4f8d-acbc-cbad4a36030d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:46.355764 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d715ec-751b-4f8d-acbc-cbad4a36030d admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:46.356049 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d715ec-751b-4f8d-acbc-cbad4a36030d admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:46.356181 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d114764-00cf-4f66-8e67-92f214cfb486 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:46.356371 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d715ec-751b-4f8d-acbc-cbad4a36030d admin admin] PATH_INFO: `/v3/projects` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:46.357098 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d715ec-751b-4f8d-acbc-cbad4a36030d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:46.359262 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d715ec-751b-4f8d-acbc-cbad4a36030d admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:46.368772 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d114764-00cf-4f66-8e67-92f214cfb486 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:46.374144 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89305a53-2054-4514-ac65-6b87a03048fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:46.374791 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89305a53-2054-4514-ac65-6b87a03048fd admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:46.375073 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89305a53-2054-4514-ac65-6b87a03048fd admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:46.375337 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89305a53-2054-4514-ac65-6b87a03048fd admin admin] PATH_INFO: `/v3/domains` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:46.375986 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89305a53-2054-4514-ac65-6b87a03048fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:58:46.376283 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89305a53-2054-4514-ac65-6b87a03048fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:46.377935 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89305a53-2054-4514-ac65-6b87a03048fd admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:46.388966 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:46.392957 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1078/2145] 10.222.0.55 () {58 vars in 1153 bytes} [Wed Aug 24 08:58:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:58:46.407465 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89305a53-2054-4514-ac65-6b87a03048fd admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:46.423528 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89305a53-2054-4514-ac65-6b87a03048fd admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:46.430937 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae9fbe9-878e-4af0-92f6-c7dbb753bb2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:46.431651 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae9fbe9-878e-4af0-92f6-c7dbb753bb2c admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:46.431934 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae9fbe9-878e-4af0-92f6-c7dbb753bb2c admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:46.432213 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae9fbe9-878e-4af0-92f6-c7dbb753bb2c admin admin] PATH_INFO: `/v3/projects` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:46.433238 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae9fbe9-878e-4af0-92f6-c7dbb753bb2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:46.437000 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae9fbe9-878e-4af0-92f6-c7dbb753bb2c admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:46.496357 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1068/2146] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 08:58:46 2022] POST /identity/v3/projects => generated 363 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:58:46.500842 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d715ec-751b-4f8d-acbc-cbad4a36030d admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:46.571701 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d715ec-751b-4f8d-acbc-cbad4a36030d admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:46.577135 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31a3893-7e18-4581-b752-a3037ed48c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:46.577886 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31a3893-7e18-4581-b752-a3037ed48c81 admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:46.578166 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31a3893-7e18-4581-b752-a3037ed48c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:46.578942 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31a3893-7e18-4581-b752-a3037ed48c81 admin admin] PATH_INFO: `/v3/users` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:46.579599 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31a3893-7e18-4581-b752-a3037ed48c81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:46.581757 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31a3893-7e18-4581-b752-a3037ed48c81 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:46.616863 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1079/2147] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 08:58:46 2022] POST /identity/v3/projects => generated 383 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:58:46.621615 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae9fbe9-878e-4af0-92f6-c7dbb753bb2c admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:46.693402 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae9fbe9-878e-4af0-92f6-c7dbb753bb2c admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:46.698339 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094a924d-afb7-4a0a-8470-2bb566c9751e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:46.698946 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094a924d-afb7-4a0a-8470-2bb566c9751e admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:46.699222 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094a924d-afb7-4a0a-8470-2bb566c9751e admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:46.699492 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094a924d-afb7-4a0a-8470-2bb566c9751e admin admin] PATH_INFO: `/v3/users` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:46.700124 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094a924d-afb7-4a0a-8470-2bb566c9751e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:46.702112 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094a924d-afb7-4a0a-8470-2bb566c9751e admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:46.719150 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:46.773441 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1069/2148] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 08:58:46 2022] POST /identity/v3/users => generated 282 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:58:46.774901 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31a3893-7e18-4581-b752-a3037ed48c81 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:46.789216 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31a3893-7e18-4581-b752-a3037ed48c81 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:46.795367 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe75985-0eaf-4b40-989b-914bfbf2462e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:46.797310 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe75985-0eaf-4b40-989b-914bfbf2462e admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:46.797795 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe75985-0eaf-4b40-989b-914bfbf2462e admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:46.798401 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe75985-0eaf-4b40-989b-914bfbf2462e admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:46.799313 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe75985-0eaf-4b40-989b-914bfbf2462e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:46.801075 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe75985-0eaf-4b40-989b-914bfbf2462e admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:46.813388 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1070/2149] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:58:46 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 08:58:46.818427 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1080/2150] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 08:58:46 2022] POST /identity/v3/users => generated 292 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:58:46.822822 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe75985-0eaf-4b40-989b-914bfbf2462e admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:46.823055 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094a924d-afb7-4a0a-8470-2bb566c9751e admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:46.834637 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe75985-0eaf-4b40-989b-914bfbf2462e admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:46.835651 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094a924d-afb7-4a0a-8470-2bb566c9751e admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:46.839834 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c7c99e-0891-42ff-b47a-272e45218210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:46.840310 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c7c99e-0891-42ff-b47a-272e45218210 admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:46.840625 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c7c99e-0891-42ff-b47a-272e45218210 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:46.840993 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c7c99e-0891-42ff-b47a-272e45218210 admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:46.842205 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c7c99e-0891-42ff-b47a-272e45218210 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:46.844995 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c7c99e-0891-42ff-b47a-272e45218210 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:46.846452 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9324a84-9bad-4697-95cd-b3910210c913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:46.848855 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9324a84-9bad-4697-95cd-b3910210c913 admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:46.849719 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9324a84-9bad-4697-95cd-b3910210c913 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:46.854279 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9324a84-9bad-4697-95cd-b3910210c913 admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:46.854279 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9324a84-9bad-4697-95cd-b3910210c913 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:46.854279 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9324a84-9bad-4697-95cd-b3910210c913 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:46.863871 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1071/2151] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:58:46 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:58:46.865160 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c7c99e-0891-42ff-b47a-272e45218210 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:46.868598 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1081/2152] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:58:46 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:58:46.871779 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9324a84-9bad-4697-95cd-b3910210c913 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:46.876682 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c7c99e-0891-42ff-b47a-272e45218210 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:46.882677 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9324a84-9bad-4697-95cd-b3910210c913 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:46.883384 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e91dbd-3280-4f0f-8eaa-fa637d413b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:46.884285 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e91dbd-3280-4f0f-8eaa-fa637d413b6b admin admin] REQUEST_METHOD: `PUT` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:46.884793 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e91dbd-3280-4f0f-8eaa-fa637d413b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:46.885204 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e91dbd-3280-4f0f-8eaa-fa637d413b6b admin admin] PATH_INFO: `/v3/projects/d348b19bf2fe46338bb60d6ca069efc0/users/9ed899a55f8a406196b984970994280d/roles/b674fd0a592e4e0cada811bba5ceea4d` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:46.887961 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fec22f4-3174-4866-958b-0431e1ef9c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:46.888599 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fec22f4-3174-4866-958b-0431e1ef9c21 admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:46.888884 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fec22f4-3174-4866-958b-0431e1ef9c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:46.889160 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fec22f4-3174-4866-958b-0431e1ef9c21 admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:46.889841 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fec22f4-3174-4866-958b-0431e1ef9c21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:46.892012 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fec22f4-3174-4866-958b-0431e1ef9c21 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:46.905423 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e91dbd-3280-4f0f-8eaa-fa637d413b6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d348b19bf2fe46338bb60d6ca069efc0, user_id=9ed899a55f8a406196b984970994280d, role_id=b674fd0a592e4e0cada811bba5ceea4d)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:46.908262 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e91dbd-3280-4f0f-8eaa-fa637d413b6b admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:46.911644 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1082/2153] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:58:46 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:58:46.916716 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fec22f4-3174-4866-958b-0431e1ef9c21 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:46.928104 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fec22f4-3174-4866-958b-0431e1ef9c21 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:46.936251 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5d7df9-33ec-47c7-ac8a-646275a64ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:46.936251 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5d7df9-33ec-47c7-ac8a-646275a64ef8 admin admin] REQUEST_METHOD: `PUT` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:46.936251 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5d7df9-33ec-47c7-ac8a-646275a64ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:46.936251 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5d7df9-33ec-47c7-ac8a-646275a64ef8 admin admin] PATH_INFO: `/v3/projects/52f074b18ac34245862d64415edd8868/users/86f2133cd9c7487bbaa95505c47ecdbb/roles/b674fd0a592e4e0cada811bba5ceea4d` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:46.953075 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5d7df9-33ec-47c7-ac8a-646275a64ef8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52f074b18ac34245862d64415edd8868, user_id=86f2133cd9c7487bbaa95505c47ecdbb, role_id=b674fd0a592e4e0cada811bba5ceea4d)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:46.955813 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5d7df9-33ec-47c7-ac8a-646275a64ef8 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:47.019552 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1072/2154] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 08:58:46 2022] PUT /identity/v3/projects/d348b19bf2fe46338bb60d6ca069efc0/users/9ed899a55f8a406196b984970994280d/roles/b674fd0a592e4e0cada811bba5ceea4d => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:58:47.031079 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdb4c76-2b3e-4328-b96e-cae54a19e516 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:47.031536 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdb4c76-2b3e-4328-b96e-cae54a19e516 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:47.031903 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdb4c76-2b3e-4328-b96e-cae54a19e516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:47.065090 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-7bdb4c76-2b3e-4328-b96e-cae54a19e516 None None] MFA Rules not processed for user `9ed899a55f8a406196b984970994280d`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:47.065571 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1083/2155] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 08:58:46 2022] PUT /identity/v3/projects/52f074b18ac34245862d64415edd8868/users/86f2133cd9c7487bbaa95505c47ecdbb/roles/b674fd0a592e4e0cada811bba5ceea4d => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:58:47.066845 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-7bdb4c76-2b3e-4328-b96e-cae54a19e516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:47.077871 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343b181d-3c83-4da5-818d-9b5cddb3737c None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:47.078173 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343b181d-3c83-4da5-818d-9b5cddb3737c None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:47.083610 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343b181d-3c83-4da5-818d-9b5cddb3737c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:47.121744 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-343b181d-3c83-4da5-818d-9b5cddb3737c None None] MFA Rules not processed for user `86f2133cd9c7487bbaa95505c47ecdbb`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:47.123755 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-343b181d-3c83-4da5-818d-9b5cddb3737c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:47.165448 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1073/2156] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:58:47 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:58:47.183309 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdb4c76-2b3e-4328-b96e-cae54a19e516 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:47.184793 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-7bdb4c76-2b3e-4328-b96e-cae54a19e516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:47.227077 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1084/2157] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:58:47 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:58:47.232492 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:47.271127 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdb4c76-2b3e-4328-b96e-cae54a19e516 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:47.277213 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817a2381-6535-435e-842b-23adf0f105d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbb9ac667f04c35a63b2689df18efed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:47.278096 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817a2381-6535-435e-842b-23adf0f105d0 service cinder] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:47.278603 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817a2381-6535-435e-842b-23adf0f105d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:47.279062 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817a2381-6535-435e-842b-23adf0f105d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:47.289531 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817a2381-6535-435e-842b-23adf0f105d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:47.290986 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817a2381-6535-435e-842b-23adf0f105d0 service cinder] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:47.301704 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1074/2158] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:58:47 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:58:47.349296 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343b181d-3c83-4da5-818d-9b5cddb3737c None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:47.350783 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-343b181d-3c83-4da5-818d-9b5cddb3737c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:47.365694 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343b181d-3c83-4da5-818d-9b5cddb3737c None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:47.372285 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd112d86-4984-4c46-8743-39c4f7e0f679 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbb9ac667f04c35a63b2689df18efed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:47.372285 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd112d86-4984-4c46-8743-39c4f7e0f679 service cinder] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:47.372285 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd112d86-4984-4c46-8743-39c4f7e0f679 service cinder] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:47.372285 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd112d86-4984-4c46-8743-39c4f7e0f679 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:47.382219 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd112d86-4984-4c46-8743-39c4f7e0f679 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:47.385404 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd112d86-4984-4c46-8743-39c4f7e0f679 service cinder] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:47.388217 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:47.396667 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1085/2159] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:58:47 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:58:51.565858 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817a2381-6535-435e-842b-23adf0f105d0 service cinder] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:51.568404 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-817a2381-6535-435e-842b-23adf0f105d0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:51.699376 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817a2381-6535-435e-842b-23adf0f105d0 service cinder] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:51.709743 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bef19b-49e5-493f-8ba3-5b0e02c90797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:51.711003 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bef19b-49e5-493f-8ba3-5b0e02c90797 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:51.711331 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bef19b-49e5-493f-8ba3-5b0e02c90797 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:51.712065 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bef19b-49e5-493f-8ba3-5b0e02c90797 admin admin] PATH_INFO: `/v3/users/9165197817204690bee2229b9d14ad56` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:51.725432 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bef19b-49e5-493f-8ba3-5b0e02c90797 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9165197817204690bee2229b9d14ad56)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:51.729859 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bef19b-49e5-493f-8ba3-5b0e02c90797 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:51.975966 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-07bef19b-49e5-493f-8ba3-5b0e02c90797 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9165197817204690bee2229b9d14ad56'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:51.976727 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-07bef19b-49e5-493f-8ba3-5b0e02c90797 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9165197817204690bee2229b9d14ad56'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:51.977522 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-07bef19b-49e5-493f-8ba3-5b0e02c90797 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9165197817204690bee2229b9d14ad56'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:51.997267 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-07bef19b-49e5-493f-8ba3-5b0e02c90797 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9165197817204690bee2229b9d14ad56'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:52.110005 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-07bef19b-49e5-493f-8ba3-5b0e02c90797 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9165197817204690bee2229b9d14ad56'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:52.130758 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1075/2160] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 08:58:51 2022] DELETE /identity/v3/users/9165197817204690bee2229b9d14ad56 => generated 0 bytes in 566 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:58:52.136816 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cbdae4-7069-4e36-b133-9154bc8040eb None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:52.137458 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cbdae4-7069-4e36-b133-9154bc8040eb None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:52.144746 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cbdae4-7069-4e36-b133-9154bc8040eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:52.168133 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-12cbdae4-7069-4e36-b133-9154bc8040eb None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:52.169629 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-12cbdae4-7069-4e36-b133-9154bc8040eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:52.257162 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1086/2161] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:58:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:58:52.271817 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bef19b-49e5-493f-8ba3-5b0e02c90797 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:52.271817 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-07bef19b-49e5-493f-8ba3-5b0e02c90797 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:52.361813 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bef19b-49e5-493f-8ba3-5b0e02c90797 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:52.365865 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1b7719-87be-42f0-936d-dad25efbb446 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:52.366501 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1b7719-87be-42f0-936d-dad25efbb446 service neutron] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:52.366785 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1b7719-87be-42f0-936d-dad25efbb446 service neutron] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:52.367105 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1b7719-87be-42f0-936d-dad25efbb446 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:52.375977 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:52.378779 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1b7719-87be-42f0-936d-dad25efbb446 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:52.380358 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1b7719-87be-42f0-936d-dad25efbb446 service neutron] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:52.391805 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1076/2162] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:58:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:58:53.306974 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cbdae4-7069-4e36-b133-9154bc8040eb None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:53.309220 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-12cbdae4-7069-4e36-b133-9154bc8040eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:53.330721 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cbdae4-7069-4e36-b133-9154bc8040eb None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:53.339255 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc77fc7-d6e5-48dd-b0b9-d24904cb1061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:53.340675 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc77fc7-d6e5-48dd-b0b9-d24904cb1061 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:53.341180 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc77fc7-d6e5-48dd-b0b9-d24904cb1061 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:53.341687 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc77fc7-d6e5-48dd-b0b9-d24904cb1061 admin admin] PATH_INFO: `/v3/projects/9137fdb1ccc3433cad5c223e0df83d21` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:53.344181 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc77fc7-d6e5-48dd-b0b9-d24904cb1061 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9137fdb1ccc3433cad5c223e0df83d21)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:53.346852 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc77fc7-d6e5-48dd-b0b9-d24904cb1061 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:53.516209 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-bcc77fc7-d6e5-48dd-b0b9-d24904cb1061 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9137fdb1ccc3433cad5c223e0df83d21', 'request_id': 'req-bcc77fc7-d6e5-48dd-b0b9-d24904cb1061'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:53.522827 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-bcc77fc7-d6e5-48dd-b0b9-d24904cb1061 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9137fdb1ccc3433cad5c223e0df83d21', 'request_id': 'req-bcc77fc7-d6e5-48dd-b0b9-d24904cb1061'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:53.620111 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-bcc77fc7-d6e5-48dd-b0b9-d24904cb1061 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9137fdb1ccc3433cad5c223e0df83d21', 'request_id': 'req-bcc77fc7-d6e5-48dd-b0b9-d24904cb1061'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:53.634470 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-bcc77fc7-d6e5-48dd-b0b9-d24904cb1061 admin admin] The token cache is being invalidate because project 9137fdb1ccc3433cad5c223e0df83d21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:58:53.634927 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-bcc77fc7-d6e5-48dd-b0b9-d24904cb1061 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:53.635715 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-bcc77fc7-d6e5-48dd-b0b9-d24904cb1061 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:53.637436 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1087/2163] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 08:58:53 2022] DELETE /identity/v3/projects/9137fdb1ccc3433cad5c223e0df83d21 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:58:53.641820 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1b7719-87be-42f0-936d-dad25efbb446 service neutron] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:53.652250 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1b7719-87be-42f0-936d-dad25efbb446 service neutron] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:53.721397 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e489ff89-5516-4467-9af1-3be4c3920ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:53.722035 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e489ff89-5516-4467-9af1-3be4c3920ccc admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:53.722312 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e489ff89-5516-4467-9af1-3be4c3920ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:53.722597 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e489ff89-5516-4467-9af1-3be4c3920ccc admin admin] PATH_INFO: `/v3/users/c8385eb0e5eb40b09c384b41c4af5f83` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:53.723821 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e489ff89-5516-4467-9af1-3be4c3920ccc admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8385eb0e5eb40b09c384b41c4af5f83)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:53.725827 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e489ff89-5516-4467-9af1-3be4c3920ccc admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:54.005205 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-e489ff89-5516-4467-9af1-3be4c3920ccc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8385eb0e5eb40b09c384b41c4af5f83'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:54.005896 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-e489ff89-5516-4467-9af1-3be4c3920ccc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8385eb0e5eb40b09c384b41c4af5f83'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:54.006523 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-e489ff89-5516-4467-9af1-3be4c3920ccc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8385eb0e5eb40b09c384b41c4af5f83'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:54.023223 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-e489ff89-5516-4467-9af1-3be4c3920ccc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8385eb0e5eb40b09c384b41c4af5f83'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:54.205125 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-e489ff89-5516-4467-9af1-3be4c3920ccc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8385eb0e5eb40b09c384b41c4af5f83'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:54.220637 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1077/2164] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 08:58:53 2022] DELETE /identity/v3/users/c8385eb0e5eb40b09c384b41c4af5f83 => generated 0 bytes in 579 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:58:54.773282 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc77fc7-d6e5-48dd-b0b9-d24904cb1061 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:54.775375 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-bcc77fc7-d6e5-48dd-b0b9-d24904cb1061 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:54.797584 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc77fc7-d6e5-48dd-b0b9-d24904cb1061 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:54.870205 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6167ad6-6b73-40ef-8944-f3307e3a9e2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:54.870857 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6167ad6-6b73-40ef-8944-f3307e3a9e2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:54.871135 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6167ad6-6b73-40ef-8944-f3307e3a9e2f admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:54.871405 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6167ad6-6b73-40ef-8944-f3307e3a9e2f admin admin] PATH_INFO: `/v3/projects/1f961c8e09ee48e191b03b6db5b5302b` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:54.872637 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6167ad6-6b73-40ef-8944-f3307e3a9e2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f961c8e09ee48e191b03b6db5b5302b)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:54.874728 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6167ad6-6b73-40ef-8944-f3307e3a9e2f admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:54.903986 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:55.014382 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-e6167ad6-6b73-40ef-8944-f3307e3a9e2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f961c8e09ee48e191b03b6db5b5302b', 'request_id': 'req-e6167ad6-6b73-40ef-8944-f3307e3a9e2f'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:55.020895 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-e6167ad6-6b73-40ef-8944-f3307e3a9e2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f961c8e09ee48e191b03b6db5b5302b', 'request_id': 'req-e6167ad6-6b73-40ef-8944-f3307e3a9e2f'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:55.109376 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-e6167ad6-6b73-40ef-8944-f3307e3a9e2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f961c8e09ee48e191b03b6db5b5302b', 'request_id': 'req-e6167ad6-6b73-40ef-8944-f3307e3a9e2f'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:55.123264 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-e6167ad6-6b73-40ef-8944-f3307e3a9e2f admin admin] The token cache is being invalidate because project 1f961c8e09ee48e191b03b6db5b5302b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:58:55.123578 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-e6167ad6-6b73-40ef-8944-f3307e3a9e2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:55.124337 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-e6167ad6-6b73-40ef-8944-f3307e3a9e2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:55.125912 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1088/2165] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 08:58:54 2022] DELETE /identity/v3/projects/1f961c8e09ee48e191b03b6db5b5302b => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:58:55.130542 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e489ff89-5516-4467-9af1-3be4c3920ccc admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:55.133102 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-e489ff89-5516-4467-9af1-3be4c3920ccc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:55.147623 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e489ff89-5516-4467-9af1-3be4c3920ccc admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:55.250917 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10d56b3-a2f9-49b7-9b07-b003d8c5a375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:55.252052 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10d56b3-a2f9-49b7-9b07-b003d8c5a375 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:55.252596 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10d56b3-a2f9-49b7-9b07-b003d8c5a375 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:55.253066 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10d56b3-a2f9-49b7-9b07-b003d8c5a375 admin admin] PATH_INFO: `/v3/users/43507f98d0c4407e87d00cdbb8c7cd7f` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:55.255338 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10d56b3-a2f9-49b7-9b07-b003d8c5a375 admin admin] RBAC: Authorizing `identity:delete_user(user_id=43507f98d0c4407e87d00cdbb8c7cd7f)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:55.259263 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10d56b3-a2f9-49b7-9b07-b003d8c5a375 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:55.284855 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:55.555009 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-d10d56b3-a2f9-49b7-9b07-b003d8c5a375 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43507f98d0c4407e87d00cdbb8c7cd7f'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:55.555754 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-d10d56b3-a2f9-49b7-9b07-b003d8c5a375 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43507f98d0c4407e87d00cdbb8c7cd7f'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:55.556419 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-d10d56b3-a2f9-49b7-9b07-b003d8c5a375 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43507f98d0c4407e87d00cdbb8c7cd7f'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:55.570282 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-d10d56b3-a2f9-49b7-9b07-b003d8c5a375 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43507f98d0c4407e87d00cdbb8c7cd7f'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:55.668836 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-d10d56b3-a2f9-49b7-9b07-b003d8c5a375 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43507f98d0c4407e87d00cdbb8c7cd7f'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:55.684190 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1078/2166] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 08:58:55 2022] DELETE /identity/v3/users/43507f98d0c4407e87d00cdbb8c7cd7f => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:58:56.173381 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6167ad6-6b73-40ef-8944-f3307e3a9e2f admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:56.175226 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-e6167ad6-6b73-40ef-8944-f3307e3a9e2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:56.196897 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6167ad6-6b73-40ef-8944-f3307e3a9e2f admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:56.280599 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9f18aa-3fdb-423b-8b07-541d463af7e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:56.281266 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9f18aa-3fdb-423b-8b07-541d463af7e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:56.281555 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9f18aa-3fdb-423b-8b07-541d463af7e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:56.281830 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9f18aa-3fdb-423b-8b07-541d463af7e4 admin admin] PATH_INFO: `/v3/projects/c0bf4f682a334b679f3d0ce761bcbbc8` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:56.283028 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9f18aa-3fdb-423b-8b07-541d463af7e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0bf4f682a334b679f3d0ce761bcbbc8)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:56.285168 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9f18aa-3fdb-423b-8b07-541d463af7e4 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:56.453403 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-5e9f18aa-3fdb-423b-8b07-541d463af7e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0bf4f682a334b679f3d0ce761bcbbc8', 'request_id': 'req-5e9f18aa-3fdb-423b-8b07-541d463af7e4'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:56.462229 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-5e9f18aa-3fdb-423b-8b07-541d463af7e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0bf4f682a334b679f3d0ce761bcbbc8', 'request_id': 'req-5e9f18aa-3fdb-423b-8b07-541d463af7e4'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:56.557506 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-5e9f18aa-3fdb-423b-8b07-541d463af7e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0bf4f682a334b679f3d0ce761bcbbc8', 'request_id': 'req-5e9f18aa-3fdb-423b-8b07-541d463af7e4'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:56.580455 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-5e9f18aa-3fdb-423b-8b07-541d463af7e4 admin admin] The token cache is being invalidate because project c0bf4f682a334b679f3d0ce761bcbbc8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:58:56.580982 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-5e9f18aa-3fdb-423b-8b07-541d463af7e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:56.582121 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-5e9f18aa-3fdb-423b-8b07-541d463af7e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:56.584830 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1089/2167] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 08:58:56 2022] DELETE /identity/v3/projects/c0bf4f682a334b679f3d0ce761bcbbc8 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:58:56.590787 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae16efc6-4227-467a-b6e2-224c9c2523ab None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:56.591637 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae16efc6-4227-467a-b6e2-224c9c2523ab None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:56.591852 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae16efc6-4227-467a-b6e2-224c9c2523ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:56.617380 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-ae16efc6-4227-467a-b6e2-224c9c2523ab None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:56.618864 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-ae16efc6-4227-467a-b6e2-224c9c2523ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:56.713561 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1079/2168] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:58:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:58:56.723995 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a791169a-55d1-4f31-8115-4f8974f8aeb0 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:56.724500 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a791169a-55d1-4f31-8115-4f8974f8aeb0 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:56.724991 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a791169a-55d1-4f31-8115-4f8974f8aeb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:56.762863 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-a791169a-55d1-4f31-8115-4f8974f8aeb0 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:56.764183 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-a791169a-55d1-4f31-8115-4f8974f8aeb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:56.778985 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1090/2169] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:58:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:58:56.784769 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae16efc6-4227-467a-b6e2-224c9c2523ab None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:56.796480 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae16efc6-4227-467a-b6e2-224c9c2523ab None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:56.804140 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8a579b-3782-47ef-bb8e-4716052bdda1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:56.804837 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8a579b-3782-47ef-bb8e-4716052bdda1 admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:56.805119 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8a579b-3782-47ef-bb8e-4716052bdda1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:56.805392 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8a579b-3782-47ef-bb8e-4716052bdda1 admin admin] PATH_INFO: `/v3/domains` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:56.806063 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8a579b-3782-47ef-bb8e-4716052bdda1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:58:56.806372 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8a579b-3782-47ef-bb8e-4716052bdda1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:56.808144 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8a579b-3782-47ef-bb8e-4716052bdda1 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:56.822133 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1080/2170] 10.222.0.55 () {58 vars in 1153 bytes} [Wed Aug 24 08:58:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:58:56.828813 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a791169a-55d1-4f31-8115-4f8974f8aeb0 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:56.839128 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a791169a-55d1-4f31-8115-4f8974f8aeb0 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:56.844335 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c26d510-8a08-4cdc-968b-da4f27e14642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:56.845001 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c26d510-8a08-4cdc-968b-da4f27e14642 admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:56.845277 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c26d510-8a08-4cdc-968b-da4f27e14642 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:56.845591 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c26d510-8a08-4cdc-968b-da4f27e14642 admin admin] PATH_INFO: `/v3/projects` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:56.849391 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c26d510-8a08-4cdc-968b-da4f27e14642 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:56.849391 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c26d510-8a08-4cdc-968b-da4f27e14642 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:56.990048 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1091/2171] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 08:58:56 2022] POST /identity/v3/projects => generated 379 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:58:56.996716 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8a579b-3782-47ef-bb8e-4716052bdda1 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:57.013990 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8a579b-3782-47ef-bb8e-4716052bdda1 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:57.100242 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ce5855-0406-446b-85fa-de035eed65f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:57.101327 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ce5855-0406-446b-85fa-de035eed65f2 admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:57.101824 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ce5855-0406-446b-85fa-de035eed65f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:57.102313 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ce5855-0406-446b-85fa-de035eed65f2 admin admin] PATH_INFO: `/v3/users` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:57.103481 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ce5855-0406-446b-85fa-de035eed65f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:57.107344 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ce5855-0406-446b-85fa-de035eed65f2 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:57.304091 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1081/2172] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 08:58:56 2022] POST /identity/v3/users => generated 290 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:58:57.310361 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c26d510-8a08-4cdc-968b-da4f27e14642 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:57.327293 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c26d510-8a08-4cdc-968b-da4f27e14642 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:57.335187 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444a66d3-b4fc-45af-bb17-13d47bff663d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:57.336191 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444a66d3-b4fc-45af-bb17-13d47bff663d admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:57.336698 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444a66d3-b4fc-45af-bb17-13d47bff663d admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:57.337188 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444a66d3-b4fc-45af-bb17-13d47bff663d admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:57.338174 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444a66d3-b4fc-45af-bb17-13d47bff663d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:57.341079 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444a66d3-b4fc-45af-bb17-13d47bff663d admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:57.362168 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1092/2173] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:58:57 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 08:58:57.363585 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ce5855-0406-446b-85fa-de035eed65f2 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:57.376350 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ce5855-0406-446b-85fa-de035eed65f2 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:57.382213 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2970620c-6705-48bf-bcfb-f7a6f59f4506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:57.382993 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2970620c-6705-48bf-bcfb-f7a6f59f4506 admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:57.383433 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2970620c-6705-48bf-bcfb-f7a6f59f4506 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:57.383769 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2970620c-6705-48bf-bcfb-f7a6f59f4506 admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:57.384641 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2970620c-6705-48bf-bcfb-f7a6f59f4506 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:57.387153 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2970620c-6705-48bf-bcfb-f7a6f59f4506 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:57.394546 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:57.407132 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1082/2174] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:58:57 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:58:57.411246 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444a66d3-b4fc-45af-bb17-13d47bff663d admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:57.419673 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:57.430055 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444a66d3-b4fc-45af-bb17-13d47bff663d admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:57.438187 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbecebc-a9b4-4293-9255-daec34ea67cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:57.439430 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbecebc-a9b4-4293-9255-daec34ea67cc admin admin] REQUEST_METHOD: `PUT` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:57.439892 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbecebc-a9b4-4293-9255-daec34ea67cc admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:57.440590 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbecebc-a9b4-4293-9255-daec34ea67cc admin admin] PATH_INFO: `/v3/projects/1f01e21018a54220ba1e54c6b7bfc400/users/c2c3752027f342a489e6eb9d3b540819/roles/b674fd0a592e4e0cada811bba5ceea4d` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:57.470475 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbecebc-a9b4-4293-9255-daec34ea67cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f01e21018a54220ba1e54c6b7bfc400, user_id=c2c3752027f342a489e6eb9d3b540819, role_id=b674fd0a592e4e0cada811bba5ceea4d)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:57.474094 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbecebc-a9b4-4293-9255-daec34ea67cc admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:57.515790 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2970620c-6705-48bf-bcfb-f7a6f59f4506 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:57.518683 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-2970620c-6705-48bf-bcfb-f7a6f59f4506 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:57.609370 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1093/2175] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 08:58:57 2022] PUT /identity/v3/projects/1f01e21018a54220ba1e54c6b7bfc400/users/c2c3752027f342a489e6eb9d3b540819/roles/b674fd0a592e4e0cada811bba5ceea4d => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:58:57.625807 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a5ae57-e0b3-4ace-9444-ea90a922b60d None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:57.626360 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a5ae57-e0b3-4ace-9444-ea90a922b60d None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:57.626824 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a5ae57-e0b3-4ace-9444-ea90a922b60d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:57.650315 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-d3a5ae57-e0b3-4ace-9444-ea90a922b60d None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:57.653299 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-d3a5ae57-e0b3-4ace-9444-ea90a922b60d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:57.692789 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2970620c-6705-48bf-bcfb-f7a6f59f4506 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:57.746472 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1094/2176] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:58:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:58:57.763512 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a5ae57-e0b3-4ace-9444-ea90a922b60d None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:57.766731 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-d3a5ae57-e0b3-4ace-9444-ea90a922b60d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:57.832893 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1cd42b-001f-44ef-a53d-9dc38841e60e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf0c99771661413aaa0f237af6b394ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:57.832893 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1cd42b-001f-44ef-a53d-9dc38841e60e service placement] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:57.832893 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1cd42b-001f-44ef-a53d-9dc38841e60e service placement] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:57.832893 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1cd42b-001f-44ef-a53d-9dc38841e60e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:57.840906 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-9b1cd42b-001f-44ef-a53d-9dc38841e60e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:57.866136 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1cd42b-001f-44ef-a53d-9dc38841e60e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:57.867849 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1cd42b-001f-44ef-a53d-9dc38841e60e service placement] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:57.886179 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1083/2177] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:58:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:58:57.911694 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a5ae57-e0b3-4ace-9444-ea90a922b60d None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:57.920352 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0ba3c6-be2f-49cd-ae23-81f229200485 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:57.920352 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0ba3c6-be2f-49cd-ae23-81f229200485 service neutron] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:57.920352 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0ba3c6-be2f-49cd-ae23-81f229200485 service neutron] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:57.920352 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0ba3c6-be2f-49cd-ae23-81f229200485 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:57.930370 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0ba3c6-be2f-49cd-ae23-81f229200485 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:57.932848 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0ba3c6-be2f-49cd-ae23-81f229200485 service neutron] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:57.948094 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1095/2178] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:58:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:59:06.049255 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbdcc0c-7c5e-47f1-af67-f8876a8cbc0f None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:06.050177 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbdcc0c-7c5e-47f1-af67-f8876a8cbc0f None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:06.050449 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbdcc0c-7c5e-47f1-af67-f8876a8cbc0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:06.095023 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-8dbdcc0c-7c5e-47f1-af67-f8876a8cbc0f None None] MFA Rules not processed for user `c2c3752027f342a489e6eb9d3b540819`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:59:06.096433 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-8dbdcc0c-7c5e-47f1-af67-f8876a8cbc0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:59:06.172503 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1084/2179] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:59:06 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:59:06.177243 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0ba3c6-be2f-49cd-ae23-81f229200485 service neutron] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:59:06.188770 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0ba3c6-be2f-49cd-ae23-81f229200485 service neutron] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:59:06.199652 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e10adc7-75b7-43c7-b1b6-f5d9b880a36b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:59:06.200118 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e10adc7-75b7-43c7-b1b6-f5d9b880a36b admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:06.200398 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e10adc7-75b7-43c7-b1b6-f5d9b880a36b admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:06.200697 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e10adc7-75b7-43c7-b1b6-f5d9b880a36b admin admin] PATH_INFO: `/v3/projects` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:06.202583 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e10adc7-75b7-43c7-b1b6-f5d9b880a36b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:59:06.204738 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e10adc7-75b7-43c7-b1b6-f5d9b880a36b admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:59:06.309149 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1096/2180] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 08:59:06 2022] POST /identity/v3/projects => generated 377 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:59:06.314837 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbdcc0c-7c5e-47f1-af67-f8876a8cbc0f None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:59:06.327307 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbdcc0c-7c5e-47f1-af67-f8876a8cbc0f None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:59:06.357412 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:59:06.420252 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4236883-4bf1-4d73-b143-7a47e42b7c9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:59:06.421847 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4236883-4bf1-4d73-b143-7a47e42b7c9c admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:06.422238 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4236883-4bf1-4d73-b143-7a47e42b7c9c admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:06.422965 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4236883-4bf1-4d73-b143-7a47e42b7c9c admin admin] PATH_INFO: `/v3/users` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:06.424438 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4236883-4bf1-4d73-b143-7a47e42b7c9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:59:06.428363 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4236883-4bf1-4d73-b143-7a47e42b7c9c admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:59:06.547520 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1085/2181] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 08:59:06 2022] POST /identity/v3/users => generated 295 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:59:06.551665 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e10adc7-75b7-43c7-b1b6-f5d9b880a36b admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:59:06.563238 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e10adc7-75b7-43c7-b1b6-f5d9b880a36b admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:59:06.568067 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9591d3dd-5bf8-47b0-b7d0-85dd9558b253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:59:06.568758 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9591d3dd-5bf8-47b0-b7d0-85dd9558b253 admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:06.569051 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9591d3dd-5bf8-47b0-b7d0-85dd9558b253 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:06.569327 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9591d3dd-5bf8-47b0-b7d0-85dd9558b253 admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:06.569976 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9591d3dd-5bf8-47b0-b7d0-85dd9558b253 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:59:06.571576 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9591d3dd-5bf8-47b0-b7d0-85dd9558b253 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:59:06.584008 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1097/2182] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:59:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:59:06.588464 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4236883-4bf1-4d73-b143-7a47e42b7c9c admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:59:06.598736 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4236883-4bf1-4d73-b143-7a47e42b7c9c admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:59:06.604472 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ed6dde-404b-4d1e-9bca-a00602e61aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:59:06.604674 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9591d3dd-5bf8-47b0-b7d0-85dd9558b253 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:59:06.605915 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ed6dde-404b-4d1e-9bca-a00602e61aac admin admin] REQUEST_METHOD: `PUT` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:06.605915 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ed6dde-404b-4d1e-9bca-a00602e61aac admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:06.606090 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ed6dde-404b-4d1e-9bca-a00602e61aac admin admin] PATH_INFO: `/v3/projects/19adc18f8f2c4de8b5d90c47645aa6f7/users/fc533493bf4e4f9cb3f5079d7b3ab3f2/roles/b870cd1acaa744e1ae1d58b9ac283821` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:06.606877 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-9591d3dd-5bf8-47b0-b7d0-85dd9558b253 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:59:06.622252 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9591d3dd-5bf8-47b0-b7d0-85dd9558b253 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:59:06.628494 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eba3d1-feb2-4dcf-9987-0cb84cc81fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:59:06.629304 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eba3d1-feb2-4dcf-9987-0cb84cc81fab admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:06.629556 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eba3d1-feb2-4dcf-9987-0cb84cc81fab admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:06.629879 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eba3d1-feb2-4dcf-9987-0cb84cc81fab admin admin] PATH_INFO: `/v3/users/86f2133cd9c7487bbaa95505c47ecdbb` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:06.634118 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9eba3d1-feb2-4dcf-9987-0cb84cc81fab admin admin] RBAC: Authorizing `identity:delete_user(user_id=86f2133cd9c7487bbaa95505c47ecdbb)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:59:06.634246 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ed6dde-404b-4d1e-9bca-a00602e61aac admin admin] RBAC: Authorizing `identity:create_grant(project_id=19adc18f8f2c4de8b5d90c47645aa6f7, user_id=fc533493bf4e4f9cb3f5079d7b3ab3f2, role_id=b870cd1acaa744e1ae1d58b9ac283821)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:59:06.634339 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9eba3d1-feb2-4dcf-9987-0cb84cc81fab admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:59:06.638537 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ed6dde-404b-4d1e-9bca-a00602e61aac admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:59:06.769203 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1086/2183] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 08:59:06 2022] PUT /identity/v3/projects/19adc18f8f2c4de8b5d90c47645aa6f7/users/fc533493bf4e4f9cb3f5079d7b3ab3f2/roles/b870cd1acaa744e1ae1d58b9ac283821 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:59:06.776249 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ed6dde-404b-4d1e-9bca-a00602e61aac admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:59:06.848487 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ed6dde-404b-4d1e-9bca-a00602e61aac admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:59:06.853273 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e856574e-0907-42be-a9f0-73ee15d830a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:59:06.853895 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e856574e-0907-42be-a9f0-73ee15d830a5 admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:06.854169 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e856574e-0907-42be-a9f0-73ee15d830a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:06.854440 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e856574e-0907-42be-a9f0-73ee15d830a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:06.855087 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e856574e-0907-42be-a9f0-73ee15d830a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:59:06.856717 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e856574e-0907-42be-a9f0-73ee15d830a5 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:59:06.868031 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1087/2184] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:59:06 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 08:59:06.872200 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e856574e-0907-42be-a9f0-73ee15d830a5 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:59:06.884509 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e856574e-0907-42be-a9f0-73ee15d830a5 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:59:06.890486 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078f11a4-9f3e-4040-afc6-52a7629d740e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:59:06.891234 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078f11a4-9f3e-4040-afc6-52a7629d740e admin admin] REQUEST_METHOD: `PUT` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:06.891605 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078f11a4-9f3e-4040-afc6-52a7629d740e admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:06.891882 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078f11a4-9f3e-4040-afc6-52a7629d740e admin admin] PATH_INFO: `/v3/projects/19adc18f8f2c4de8b5d90c47645aa6f7/users/fc533493bf4e4f9cb3f5079d7b3ab3f2/roles/b870cd1acaa744e1ae1d58b9ac283821` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:06.893952 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078f11a4-9f3e-4040-afc6-52a7629d740e admin admin] RBAC: Authorizing `identity:create_grant(project_id=19adc18f8f2c4de8b5d90c47645aa6f7, user_id=fc533493bf4e4f9cb3f5079d7b3ab3f2, role_id=b870cd1acaa744e1ae1d58b9ac283821)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:59:06.896729 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078f11a4-9f3e-4040-afc6-52a7629d740e admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:59:06.913100 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1088/2185] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 08:59:06 2022] PUT /identity/v3/projects/19adc18f8f2c4de8b5d90c47645aa6f7/users/fc533493bf4e4f9cb3f5079d7b3ab3f2/roles/b870cd1acaa744e1ae1d58b9ac283821 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:59:06.925085 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:59:06.928714 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-e9eba3d1-feb2-4dcf-9987-0cb84cc81fab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86f2133cd9c7487bbaa95505c47ecdbb'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:59:06.929805 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-e9eba3d1-feb2-4dcf-9987-0cb84cc81fab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86f2133cd9c7487bbaa95505c47ecdbb'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:59:06.930825 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-e9eba3d1-feb2-4dcf-9987-0cb84cc81fab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86f2133cd9c7487bbaa95505c47ecdbb'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:59:06.947814 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-e9eba3d1-feb2-4dcf-9987-0cb84cc81fab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86f2133cd9c7487bbaa95505c47ecdbb'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:59:07.188185 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-e9eba3d1-feb2-4dcf-9987-0cb84cc81fab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86f2133cd9c7487bbaa95505c47ecdbb'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:59:07.210949 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1098/2186] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 08:59:06 2022] DELETE /identity/v3/users/86f2133cd9c7487bbaa95505c47ecdbb => generated 0 bytes in 607 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:59:07.215293 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def530d1-7920-4f5d-9cb9-0dfba2c0e958 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:07.215794 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def530d1-7920-4f5d-9cb9-0dfba2c0e958 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:07.215877 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def530d1-7920-4f5d-9cb9-0dfba2c0e958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:07.237524 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-def530d1-7920-4f5d-9cb9-0dfba2c0e958 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:59:07.238913 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-def530d1-7920-4f5d-9cb9-0dfba2c0e958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:59:07.306809 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1089/2187] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:59:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:59:07.316601 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eba3d1-feb2-4dcf-9987-0cb84cc81fab admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:59:07.318071 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-e9eba3d1-feb2-4dcf-9987-0cb84cc81fab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:59:07.415055 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eba3d1-feb2-4dcf-9987-0cb84cc81fab admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:59:07.420007 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e9c602-7e29-4ffb-8bd9-b0f0eaa26bdf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:59:07.421029 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e9c602-7e29-4ffb-8bd9-b0f0eaa26bdf service neutron] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:07.421693 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e9c602-7e29-4ffb-8bd9-b0f0eaa26bdf service neutron] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:07.422131 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e9c602-7e29-4ffb-8bd9-b0f0eaa26bdf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:07.435194 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e9c602-7e29-4ffb-8bd9-b0f0eaa26bdf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:59:07.437568 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e9c602-7e29-4ffb-8bd9-b0f0eaa26bdf service neutron] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:59:07.455433 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1099/2188] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:59:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:59:08.193641 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def530d1-7920-4f5d-9cb9-0dfba2c0e958 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:59:08.196910 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-def530d1-7920-4f5d-9cb9-0dfba2c0e958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:59:08.243947 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def530d1-7920-4f5d-9cb9-0dfba2c0e958 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:59:08.254209 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8d9e8e-e306-45ab-ab7e-7177778a976b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:59:08.258027 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8d9e8e-e306-45ab-ab7e-7177778a976b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:08.258027 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8d9e8e-e306-45ab-ab7e-7177778a976b admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:08.258027 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8d9e8e-e306-45ab-ab7e-7177778a976b admin admin] PATH_INFO: `/v3/projects/52f074b18ac34245862d64415edd8868` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:08.259730 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8d9e8e-e306-45ab-ab7e-7177778a976b admin admin] RBAC: Authorizing `identity:delete_project(project_id=52f074b18ac34245862d64415edd8868)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:59:08.264382 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8d9e8e-e306-45ab-ab7e-7177778a976b admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:59:08.295613 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:59:08.436247 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-af8d9e8e-e306-45ab-ab7e-7177778a976b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52f074b18ac34245862d64415edd8868', 'request_id': 'req-af8d9e8e-e306-45ab-ab7e-7177778a976b'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:59:08.513443 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-af8d9e8e-e306-45ab-ab7e-7177778a976b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52f074b18ac34245862d64415edd8868', 'request_id': 'req-af8d9e8e-e306-45ab-ab7e-7177778a976b'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:59:08.525360 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-af8d9e8e-e306-45ab-ab7e-7177778a976b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52f074b18ac34245862d64415edd8868', 'request_id': 'req-af8d9e8e-e306-45ab-ab7e-7177778a976b'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:59:08.539821 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-af8d9e8e-e306-45ab-ab7e-7177778a976b admin admin] The token cache is being invalidate because project 52f074b18ac34245862d64415edd8868 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:59:08.540226 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-af8d9e8e-e306-45ab-ab7e-7177778a976b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:59:08.540991 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-af8d9e8e-e306-45ab-ab7e-7177778a976b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:59:08.542949 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1090/2189] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 08:59:08 2022] DELETE /identity/v3/projects/52f074b18ac34245862d64415edd8868 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:59:08.547908 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeb4451-78a8-4bb4-917b-2412fc8df2b2 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:08.548363 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeb4451-78a8-4bb4-917b-2412fc8df2b2 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:08.548456 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeb4451-78a8-4bb4-917b-2412fc8df2b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:08.572283 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-deeb4451-78a8-4bb4-917b-2412fc8df2b2 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:59:08.573821 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-deeb4451-78a8-4bb4-917b-2412fc8df2b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:59:08.643066 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1100/2190] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:59:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:59:08.651180 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dc4e99-c898-425a-b181-7f894fe23803 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:08.651475 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dc4e99-c898-425a-b181-7f894fe23803 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:08.651754 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dc4e99-c898-425a-b181-7f894fe23803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:08.673324 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-20dc4e99-c898-425a-b181-7f894fe23803 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:59:08.674557 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-20dc4e99-c898-425a-b181-7f894fe23803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:59:08.682630 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1091/2191] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:59:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:59:08.687411 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeb4451-78a8-4bb4-917b-2412fc8df2b2 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:59:08.697347 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeb4451-78a8-4bb4-917b-2412fc8df2b2 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:59:08.702781 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f56491-52ad-43ca-a1dc-5e22cb252ba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:59:08.703381 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f56491-52ad-43ca-a1dc-5e22cb252ba0 admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:08.703660 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f56491-52ad-43ca-a1dc-5e22cb252ba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:08.703995 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f56491-52ad-43ca-a1dc-5e22cb252ba0 admin admin] PATH_INFO: `/v3/domains` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:08.704678 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f56491-52ad-43ca-a1dc-5e22cb252ba0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:59:08.704983 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f56491-52ad-43ca-a1dc-5e22cb252ba0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:59:08.706582 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f56491-52ad-43ca-a1dc-5e22cb252ba0 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:59:08.718668 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1101/2192] 10.222.0.55 () {58 vars in 1153 bytes} [Wed Aug 24 08:59:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:59:08.723078 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dc4e99-c898-425a-b181-7f894fe23803 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:59:08.732512 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dc4e99-c898-425a-b181-7f894fe23803 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:59:08.737434 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eef75bc-27bb-4b09-935e-4f41ced9b0d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:59:08.737976 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eef75bc-27bb-4b09-935e-4f41ced9b0d9 admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:08.738382 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eef75bc-27bb-4b09-935e-4f41ced9b0d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:08.738607 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eef75bc-27bb-4b09-935e-4f41ced9b0d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:08.739245 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eef75bc-27bb-4b09-935e-4f41ced9b0d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:59:08.741213 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eef75bc-27bb-4b09-935e-4f41ced9b0d9 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:59:08.852780 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1092/2193] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 08:59:08 2022] POST /identity/v3/projects => generated 371 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:59:08.856906 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f56491-52ad-43ca-a1dc-5e22cb252ba0 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:59:08.868689 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f56491-52ad-43ca-a1dc-5e22cb252ba0 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:59:08.949605 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:59:08.965107 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13dcbc3-4c30-447d-b4f8-21b20c1d79e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:59:08.965729 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13dcbc3-4c30-447d-b4f8-21b20c1d79e3 admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:08.976214 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13dcbc3-4c30-447d-b4f8-21b20c1d79e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:08.976214 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13dcbc3-4c30-447d-b4f8-21b20c1d79e3 admin admin] PATH_INFO: `/v3/users` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:08.976214 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13dcbc3-4c30-447d-b4f8-21b20c1d79e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:59:08.976214 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13dcbc3-4c30-447d-b4f8-21b20c1d79e3 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:59:09.073745 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1102/2194] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 08:59:08 2022] POST /identity/v3/users => generated 286 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:59:09.077602 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eef75bc-27bb-4b09-935e-4f41ced9b0d9 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:59:09.095642 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eef75bc-27bb-4b09-935e-4f41ced9b0d9 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:59:09.108591 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e3118e-a52d-49ee-9a4e-b3a6f7cfbb4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:59:09.115248 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e3118e-a52d-49ee-9a4e-b3a6f7cfbb4f admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:09.117612 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e3118e-a52d-49ee-9a4e-b3a6f7cfbb4f admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:09.130893 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e3118e-a52d-49ee-9a4e-b3a6f7cfbb4f admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:09.130893 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e3118e-a52d-49ee-9a4e-b3a6f7cfbb4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:59:09.131853 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e3118e-a52d-49ee-9a4e-b3a6f7cfbb4f admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:59:09.156613 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1093/2195] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:59:09 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 08:59:09.168227 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13dcbc3-4c30-447d-b4f8-21b20c1d79e3 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:59:09.181681 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13dcbc3-4c30-447d-b4f8-21b20c1d79e3 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:59:09.192281 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4248ecab-3f43-47b7-8c27-9c6e43b86712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:59:09.199099 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4248ecab-3f43-47b7-8c27-9c6e43b86712 admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:09.199746 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4248ecab-3f43-47b7-8c27-9c6e43b86712 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:09.210400 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4248ecab-3f43-47b7-8c27-9c6e43b86712 admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:09.211512 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4248ecab-3f43-47b7-8c27-9c6e43b86712 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:59:09.213574 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4248ecab-3f43-47b7-8c27-9c6e43b86712 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:59:09.234927 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1103/2196] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 08:59:09 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:59:09.239497 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e3118e-a52d-49ee-9a4e-b3a6f7cfbb4f admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:59:09.257862 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e3118e-a52d-49ee-9a4e-b3a6f7cfbb4f admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:59:09.267797 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c1a868-34b2-4d0c-9e95-f0f304fdc004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:59:09.268505 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c1a868-34b2-4d0c-9e95-f0f304fdc004 admin admin] REQUEST_METHOD: `PUT` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:09.268802 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c1a868-34b2-4d0c-9e95-f0f304fdc004 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:09.269240 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c1a868-34b2-4d0c-9e95-f0f304fdc004 admin admin] PATH_INFO: `/v3/projects/e8150111178443a7918fd85afead2e6e/users/414f62aa02f54bf9a2263a68d3a4d00c/roles/b674fd0a592e4e0cada811bba5ceea4d` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:09.293103 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c1a868-34b2-4d0c-9e95-f0f304fdc004 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8150111178443a7918fd85afead2e6e, user_id=414f62aa02f54bf9a2263a68d3a4d00c, role_id=b674fd0a592e4e0cada811bba5ceea4d)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:59:09.296826 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c1a868-34b2-4d0c-9e95-f0f304fdc004 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:59:09.508649 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1094/2197] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 08:59:09 2022] PUT /identity/v3/projects/e8150111178443a7918fd85afead2e6e/users/414f62aa02f54bf9a2263a68d3a4d00c/roles/b674fd0a592e4e0cada811bba5ceea4d => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:59:09.521323 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa7776a-58d4-485c-8c2c-51e10e1c7f62 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:09.521834 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa7776a-58d4-485c-8c2c-51e10e1c7f62 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:09.522315 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa7776a-58d4-485c-8c2c-51e10e1c7f62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:09.597605 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-7aa7776a-58d4-485c-8c2c-51e10e1c7f62 None None] MFA Rules not processed for user `414f62aa02f54bf9a2263a68d3a4d00c`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:59:09.599524 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-7aa7776a-58d4-485c-8c2c-51e10e1c7f62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:59:09.719337 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1104/2198] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:59:09 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:59:09.731637 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c1a868-34b2-4d0c-9e95-f0f304fdc004 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:59:09.733572 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-67c1a868-34b2-4d0c-9e95-f0f304fdc004 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:59:09.845813 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c1a868-34b2-4d0c-9e95-f0f304fdc004 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:59:09.852303 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720b0edf-96e3-44b6-8614-4c476a56a3b6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5512043ca9a7421b821741ca88d50b07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:59:09.853019 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720b0edf-96e3-44b6-8614-4c476a56a3b6 service glance] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:09.853464 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720b0edf-96e3-44b6-8614-4c476a56a3b6 service glance] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:09.853916 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720b0edf-96e3-44b6-8614-4c476a56a3b6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:09.871669 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720b0edf-96e3-44b6-8614-4c476a56a3b6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:59:09.875392 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720b0edf-96e3-44b6-8614-4c476a56a3b6 service glance] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:59:09.894161 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1095/2199] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:59:09 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:59:14.737796 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9763869b-940c-4038-a783-dedb6a1b05f5 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:14.738347 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9763869b-940c-4038-a783-dedb6a1b05f5 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:14.738731 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9763869b-940c-4038-a783-dedb6a1b05f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:14.800370 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-9763869b-940c-4038-a783-dedb6a1b05f5 None None] MFA Rules not processed for user `fc533493bf4e4f9cb3f5079d7b3ab3f2`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:59:14.801853 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-9763869b-940c-4038-a783-dedb6a1b05f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:59:14.906580 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1105/2200] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:59:14 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:59:14.927294 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720b0edf-96e3-44b6-8614-4c476a56a3b6 service glance] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:59:14.929124 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-720b0edf-96e3-44b6-8614-4c476a56a3b6 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:59:14.985644 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:59:15.036703 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720b0edf-96e3-44b6-8614-4c476a56a3b6 service glance] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:59:15.043034 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e0a5c1-2d43-45bc-9b57-dac2ce746c7e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17da357e555420799ee681bffbcc268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:59:15.043801 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e0a5c1-2d43-45bc-9b57-dac2ce746c7e service nova] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:15.044620 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e0a5c1-2d43-45bc-9b57-dac2ce746c7e service nova] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:15.045491 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e0a5c1-2d43-45bc-9b57-dac2ce746c7e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:15.048949 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-b8e0a5c1-2d43-45bc-9b57-dac2ce746c7e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:59:15.124621 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e0a5c1-2d43-45bc-9b57-dac2ce746c7e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:59:15.128629 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e0a5c1-2d43-45bc-9b57-dac2ce746c7e service nova] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:59:15.139647 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1096/2201] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 08:59:14 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:59:19.318515 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b843ecd-ffee-4258-9012-752f3f4de7c3 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:19.318985 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b843ecd-ffee-4258-9012-752f3f4de7c3 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:19.319080 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b843ecd-ffee-4258-9012-752f3f4de7c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:19.340321 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-8b843ecd-ffee-4258-9012-752f3f4de7c3 None None] MFA Rules not processed for user `c11f1ac1772b46d2a03165571edb6f10`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:59:19.341573 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-8b843ecd-ffee-4258-9012-752f3f4de7c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:59:19.390187 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:59:19.406737 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1106/2202] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 08:59:19 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:59:25.134981 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=41 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:59:29.602762 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=5 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:00:15.914691 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8155637-eb1c-4537-96a1-cbc27868b047 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:15.915306 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8155637-eb1c-4537-96a1-cbc27868b047 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:15.915306 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8155637-eb1c-4537-96a1-cbc27868b047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:15.935096 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-c8155637-eb1c-4537-96a1-cbc27868b047 None None] MFA Rules not processed for user `c2c3752027f342a489e6eb9d3b540819`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:00:15.936305 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-c8155637-eb1c-4537-96a1-cbc27868b047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:15.943241 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1097/2203] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:00:15 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:00:21.716056 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e046641b-5087-4285-9773-71bc18b56e02 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:21.716888 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e046641b-5087-4285-9773-71bc18b56e02 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:21.717374 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e046641b-5087-4285-9773-71bc18b56e02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:21.740681 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-e046641b-5087-4285-9773-71bc18b56e02 None None] MFA Rules not processed for user `993697d33b7841668b1eabcdcd6b40f9`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:00:21.742255 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-e046641b-5087-4285-9773-71bc18b56e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:21.806901 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1107/2204] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:00:21 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:00:25.929669 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=3 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:00:28.017694 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8155637-eb1c-4537-96a1-cbc27868b047 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:28.020399 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-c8155637-eb1c-4537-96a1-cbc27868b047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:28.097492 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8155637-eb1c-4537-96a1-cbc27868b047 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:28.101372 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c3413e-a7cc-430e-9316-88cf15dbf17a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:28.102000 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c3413e-a7cc-430e-9316-88cf15dbf17a service neutron] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:28.102277 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c3413e-a7cc-430e-9316-88cf15dbf17a service neutron] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:28.102551 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c3413e-a7cc-430e-9316-88cf15dbf17a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:28.104627 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-23c3413e-a7cc-430e-9316-88cf15dbf17a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:28.122236 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c3413e-a7cc-430e-9316-88cf15dbf17a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:28.125220 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c3413e-a7cc-430e-9316-88cf15dbf17a service neutron] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:28.143107 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1098/2205] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:00:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:00:28.519241 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e046641b-5087-4285-9773-71bc18b56e02 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:28.520898 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-e046641b-5087-4285-9773-71bc18b56e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:28.534231 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e046641b-5087-4285-9773-71bc18b56e02 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:28.538321 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b176f4-adf2-4867-87bf-33e8e02dcb1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:28.538948 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b176f4-adf2-4867-87bf-33e8e02dcb1d service neutron] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:28.539228 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b176f4-adf2-4867-87bf-33e8e02dcb1d service neutron] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:28.539500 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b176f4-adf2-4867-87bf-33e8e02dcb1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:28.541621 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-18b176f4-adf2-4867-87bf-33e8e02dcb1d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:28.557986 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b176f4-adf2-4867-87bf-33e8e02dcb1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:28.561586 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b176f4-adf2-4867-87bf-33e8e02dcb1d service neutron] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:28.582931 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1108/2206] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:00:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:00:34.631990 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c3413e-a7cc-430e-9316-88cf15dbf17a service neutron] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:34.633699 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-23c3413e-a7cc-430e-9316-88cf15dbf17a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:34.712476 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c3413e-a7cc-430e-9316-88cf15dbf17a service neutron] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:34.718725 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960f3f13-494a-458c-b24f-3d247c928608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:34.719396 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960f3f13-494a-458c-b24f-3d247c928608 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:34.719702 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960f3f13-494a-458c-b24f-3d247c928608 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:34.719977 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960f3f13-494a-458c-b24f-3d247c928608 admin admin] PATH_INFO: `/v3/users/9ed899a55f8a406196b984970994280d` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:34.721235 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960f3f13-494a-458c-b24f-3d247c928608 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ed899a55f8a406196b984970994280d)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:34.723269 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960f3f13-494a-458c-b24f-3d247c928608 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:34.780406 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b176f4-adf2-4867-87bf-33e8e02dcb1d service neutron] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:34.793318 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b176f4-adf2-4867-87bf-33e8e02dcb1d service neutron] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:34.798344 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6709f2-7e85-4b02-8f5c-b2f039a2d79e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:34.799000 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6709f2-7e85-4b02-8f5c-b2f039a2d79e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:34.799281 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6709f2-7e85-4b02-8f5c-b2f039a2d79e admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:34.799557 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6709f2-7e85-4b02-8f5c-b2f039a2d79e admin admin] PATH_INFO: `/v3/users/414f62aa02f54bf9a2263a68d3a4d00c` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:34.800856 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6709f2-7e85-4b02-8f5c-b2f039a2d79e admin admin] RBAC: Authorizing `identity:delete_user(user_id=414f62aa02f54bf9a2263a68d3a4d00c)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:34.802972 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6709f2-7e85-4b02-8f5c-b2f039a2d79e admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:34.956853 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-960f3f13-494a-458c-b24f-3d247c928608 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ed899a55f8a406196b984970994280d'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:00:34.957770 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-960f3f13-494a-458c-b24f-3d247c928608 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ed899a55f8a406196b984970994280d'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:00:34.958667 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-960f3f13-494a-458c-b24f-3d247c928608 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ed899a55f8a406196b984970994280d'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:00:34.976148 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-960f3f13-494a-458c-b24f-3d247c928608 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ed899a55f8a406196b984970994280d'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:00:35.023915 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-3d6709f2-7e85-4b02-8f5c-b2f039a2d79e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '414f62aa02f54bf9a2263a68d3a4d00c'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:00:35.025607 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-3d6709f2-7e85-4b02-8f5c-b2f039a2d79e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '414f62aa02f54bf9a2263a68d3a4d00c'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:00:35.026807 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-3d6709f2-7e85-4b02-8f5c-b2f039a2d79e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '414f62aa02f54bf9a2263a68d3a4d00c'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:00:35.047662 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-3d6709f2-7e85-4b02-8f5c-b2f039a2d79e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '414f62aa02f54bf9a2263a68d3a4d00c'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:00:35.049823 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-960f3f13-494a-458c-b24f-3d247c928608 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ed899a55f8a406196b984970994280d'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:00:35.074745 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1099/2207] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 09:00:34 2022] DELETE /identity/v3/users/9ed899a55f8a406196b984970994280d => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:00:35.080957 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726fce03-58ff-4229-b040-d1ffffafd868 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:35.081827 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726fce03-58ff-4229-b040-d1ffffafd868 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:35.087076 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726fce03-58ff-4229-b040-d1ffffafd868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:35.113322 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-726fce03-58ff-4229-b040-d1ffffafd868 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:00:35.115792 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-726fce03-58ff-4229-b040-d1ffffafd868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:35.134847 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-3d6709f2-7e85-4b02-8f5c-b2f039a2d79e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '414f62aa02f54bf9a2263a68d3a4d00c'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:00:35.146502 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:00:35.162439 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1109/2208] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 09:00:34 2022] DELETE /identity/v3/users/414f62aa02f54bf9a2263a68d3a4d00c => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:00:35.169148 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeb20bd-c239-46c3-a128-dbe834e47f50 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:35.169450 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeb20bd-c239-46c3-a128-dbe834e47f50 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:35.169689 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeb20bd-c239-46c3-a128-dbe834e47f50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:35.196374 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-efeb20bd-c239-46c3-a128-dbe834e47f50 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:00:35.196724 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-efeb20bd-c239-46c3-a128-dbe834e47f50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:35.238555 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1100/2209] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:00:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:00:35.251323 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726fce03-58ff-4229-b040-d1ffffafd868 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:35.253619 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-726fce03-58ff-4229-b040-d1ffffafd868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:35.282089 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1110/2210] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:00:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:00:35.297048 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeb20bd-c239-46c3-a128-dbe834e47f50 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:35.298527 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-efeb20bd-c239-46c3-a128-dbe834e47f50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:35.305989 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:00:35.321565 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726fce03-58ff-4229-b040-d1ffffafd868 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:35.380404 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeb20bd-c239-46c3-a128-dbe834e47f50 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:35.389356 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9250423a-b624-4eed-93ee-9a52ba5c09be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:35.390094 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9250423a-b624-4eed-93ee-9a52ba5c09be service neutron] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:35.390440 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9250423a-b624-4eed-93ee-9a52ba5c09be service neutron] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:35.390578 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585bc05b-ec78-4c29-8c2e-4298e1b7c9df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:35.390973 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9250423a-b624-4eed-93ee-9a52ba5c09be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:35.391371 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585bc05b-ec78-4c29-8c2e-4298e1b7c9df service neutron] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:35.391838 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585bc05b-ec78-4c29-8c2e-4298e1b7c9df service neutron] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:35.392285 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585bc05b-ec78-4c29-8c2e-4298e1b7c9df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:35.404380 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9250423a-b624-4eed-93ee-9a52ba5c09be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:35.406507 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9250423a-b624-4eed-93ee-9a52ba5c09be service neutron] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:35.411146 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585bc05b-ec78-4c29-8c2e-4298e1b7c9df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:35.414065 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585bc05b-ec78-4c29-8c2e-4298e1b7c9df service neutron] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:35.422873 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1101/2211] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:00:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 09:00:35.442532 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1111/2212] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:00:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 09:00:36.390956 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9250423a-b624-4eed-93ee-9a52ba5c09be service neutron] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:36.392670 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-9250423a-b624-4eed-93ee-9a52ba5c09be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:36.407933 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9250423a-b624-4eed-93ee-9a52ba5c09be service neutron] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:36.413123 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569fb3aa-1236-43bc-9e72-dbcbe499eb55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:36.413697 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569fb3aa-1236-43bc-9e72-dbcbe499eb55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:36.413982 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569fb3aa-1236-43bc-9e72-dbcbe499eb55 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:36.414255 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569fb3aa-1236-43bc-9e72-dbcbe499eb55 admin admin] PATH_INFO: `/v3/projects/d348b19bf2fe46338bb60d6ca069efc0` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:36.415469 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569fb3aa-1236-43bc-9e72-dbcbe499eb55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d348b19bf2fe46338bb60d6ca069efc0)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:36.418102 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569fb3aa-1236-43bc-9e72-dbcbe499eb55 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:36.449150 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585bc05b-ec78-4c29-8c2e-4298e1b7c9df service neutron] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:36.451273 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-585bc05b-ec78-4c29-8c2e-4298e1b7c9df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:36.467920 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585bc05b-ec78-4c29-8c2e-4298e1b7c9df service neutron] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:36.474332 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ab6b78-d6f9-485b-abf0-b146588f3aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:36.475189 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ab6b78-d6f9-485b-abf0-b146588f3aee admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:36.475627 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ab6b78-d6f9-485b-abf0-b146588f3aee admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:36.476007 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ab6b78-d6f9-485b-abf0-b146588f3aee admin admin] PATH_INFO: `/v3/projects/e8150111178443a7918fd85afead2e6e` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:36.477824 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ab6b78-d6f9-485b-abf0-b146588f3aee admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8150111178443a7918fd85afead2e6e)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:36.481095 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ab6b78-d6f9-485b-abf0-b146588f3aee admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:36.574669 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-569fb3aa-1236-43bc-9e72-dbcbe499eb55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd348b19bf2fe46338bb60d6ca069efc0', 'request_id': 'req-569fb3aa-1236-43bc-9e72-dbcbe499eb55'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:00:36.658671 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-569fb3aa-1236-43bc-9e72-dbcbe499eb55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd348b19bf2fe46338bb60d6ca069efc0', 'request_id': 'req-569fb3aa-1236-43bc-9e72-dbcbe499eb55'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:00:36.666727 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-569fb3aa-1236-43bc-9e72-dbcbe499eb55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd348b19bf2fe46338bb60d6ca069efc0', 'request_id': 'req-569fb3aa-1236-43bc-9e72-dbcbe499eb55'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:00:36.680741 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-569fb3aa-1236-43bc-9e72-dbcbe499eb55 admin admin] The token cache is being invalidate because project d348b19bf2fe46338bb60d6ca069efc0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:00:36.681046 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-569fb3aa-1236-43bc-9e72-dbcbe499eb55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:00:36.681740 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-569fb3aa-1236-43bc-9e72-dbcbe499eb55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:00:36.684227 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1102/2213] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 09:00:36 2022] DELETE /identity/v3/projects/d348b19bf2fe46338bb60d6ca069efc0 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:00:36.689905 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-b0ab6b78-d6f9-485b-abf0-b146588f3aee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8150111178443a7918fd85afead2e6e', 'request_id': 'req-b0ab6b78-d6f9-485b-abf0-b146588f3aee'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:00:36.691013 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f515888b-a26e-4f57-9bea-edb44aba591c None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:36.691372 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f515888b-a26e-4f57-9bea-edb44aba591c None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:36.691917 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f515888b-a26e-4f57-9bea-edb44aba591c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:36.697983 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-b0ab6b78-d6f9-485b-abf0-b146588f3aee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8150111178443a7918fd85afead2e6e', 'request_id': 'req-b0ab6b78-d6f9-485b-abf0-b146588f3aee'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:00:36.706052 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:00:36.718630 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-f515888b-a26e-4f57-9bea-edb44aba591c None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:00:36.720064 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-f515888b-a26e-4f57-9bea-edb44aba591c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:36.798421 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-b0ab6b78-d6f9-485b-abf0-b146588f3aee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8150111178443a7918fd85afead2e6e', 'request_id': 'req-b0ab6b78-d6f9-485b-abf0-b146588f3aee'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:00:36.800814 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1103/2214] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:00:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:00:36.814808 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc599315-f80a-4f2e-8926-9ccb46fdbc73 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:36.815315 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc599315-f80a-4f2e-8926-9ccb46fdbc73 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:36.815813 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc599315-f80a-4f2e-8926-9ccb46fdbc73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:36.831383 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-b0ab6b78-d6f9-485b-abf0-b146588f3aee admin admin] The token cache is being invalidate because project e8150111178443a7918fd85afead2e6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:00:36.832287 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-b0ab6b78-d6f9-485b-abf0-b146588f3aee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:00:36.833601 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-b0ab6b78-d6f9-485b-abf0-b146588f3aee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:00:36.836901 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1112/2215] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 09:00:36 2022] DELETE /identity/v3/projects/e8150111178443a7918fd85afead2e6e => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:00:36.845519 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ee8f73-3d72-4780-b8dd-83b06bec8bce None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:36.845952 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ee8f73-3d72-4780-b8dd-83b06bec8bce None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:36.846320 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ee8f73-3d72-4780-b8dd-83b06bec8bce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:36.851442 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-dc599315-f80a-4f2e-8926-9ccb46fdbc73 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:00:36.853603 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-dc599315-f80a-4f2e-8926-9ccb46fdbc73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:36.866137 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1104/2216] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:00:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:00:36.872032 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc599315-f80a-4f2e-8926-9ccb46fdbc73 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:36.874926 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-a3ee8f73-3d72-4780-b8dd-83b06bec8bce None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:00:36.876280 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-a3ee8f73-3d72-4780-b8dd-83b06bec8bce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:36.897959 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc599315-f80a-4f2e-8926-9ccb46fdbc73 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:36.898556 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1113/2217] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:00:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:00:36.899591 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f48d80-7218-4cc0-b544-42b31b503fd3 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:36.899878 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f48d80-7218-4cc0-b544-42b31b503fd3 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:36.900155 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f48d80-7218-4cc0-b544-42b31b503fd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:36.900749 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b5e4e8-1f42-4713-a35d-d347da6487bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:36.901497 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b5e4e8-1f42-4713-a35d-d347da6487bb admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:36.902605 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b5e4e8-1f42-4713-a35d-d347da6487bb admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:36.903481 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b5e4e8-1f42-4713-a35d-d347da6487bb admin admin] PATH_INFO: `/v3/domains` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:36.905161 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b5e4e8-1f42-4713-a35d-d347da6487bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:00:36.905790 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b5e4e8-1f42-4713-a35d-d347da6487bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:36.909307 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b5e4e8-1f42-4713-a35d-d347da6487bb admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:36.924441 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-f9f48d80-7218-4cc0-b544-42b31b503fd3 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:00:36.926428 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-f9f48d80-7218-4cc0-b544-42b31b503fd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:36.931841 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1105/2218] 10.222.0.55 () {58 vars in 1153 bytes} [Wed Aug 24 09:00:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:00:36.934196 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b5e4e8-1f42-4713-a35d-d347da6487bb admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:36.940929 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1114/2219] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:00:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:00:36.947668 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f48d80-7218-4cc0-b544-42b31b503fd3 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:36.956202 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b5e4e8-1f42-4713-a35d-d347da6487bb admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:36.960138 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f48d80-7218-4cc0-b544-42b31b503fd3 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:36.962750 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51933822-3a1c-485a-aa7d-effccebe1c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:36.963457 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51933822-3a1c-485a-aa7d-effccebe1c59 admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:36.963845 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51933822-3a1c-485a-aa7d-effccebe1c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:36.965286 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51933822-3a1c-485a-aa7d-effccebe1c59 admin admin] PATH_INFO: `/v3/projects` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:36.965489 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51933822-3a1c-485a-aa7d-effccebe1c59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:36.966923 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adee7841-a0cf-4d6b-97b4-1b1a7f9b6e07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:36.967607 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51933822-3a1c-485a-aa7d-effccebe1c59 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:36.967807 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adee7841-a0cf-4d6b-97b4-1b1a7f9b6e07 admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:36.968015 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adee7841-a0cf-4d6b-97b4-1b1a7f9b6e07 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:36.968497 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adee7841-a0cf-4d6b-97b4-1b1a7f9b6e07 admin admin] PATH_INFO: `/v3/domains` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:36.969279 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adee7841-a0cf-4d6b-97b4-1b1a7f9b6e07 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:00:36.969653 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adee7841-a0cf-4d6b-97b4-1b1a7f9b6e07 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:36.971453 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adee7841-a0cf-4d6b-97b4-1b1a7f9b6e07 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:36.987556 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1115/2220] 10.222.0.55 () {58 vars in 1153 bytes} [Wed Aug 24 09:00:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:00:36.993429 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adee7841-a0cf-4d6b-97b4-1b1a7f9b6e07 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:37.003383 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adee7841-a0cf-4d6b-97b4-1b1a7f9b6e07 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:37.008811 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d32a929-af08-424e-8ce0-8af8203bb21d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:37.009626 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d32a929-af08-424e-8ce0-8af8203bb21d admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:37.009903 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d32a929-af08-424e-8ce0-8af8203bb21d admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:37.010179 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d32a929-af08-424e-8ce0-8af8203bb21d admin admin] PATH_INFO: `/v3/projects` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:37.010832 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d32a929-af08-424e-8ce0-8af8203bb21d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:37.012910 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d32a929-af08-424e-8ce0-8af8203bb21d admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:37.094695 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1106/2221] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 09:00:36 2022] POST /identity/v3/projects => generated 375 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:00:37.097077 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51933822-3a1c-485a-aa7d-effccebe1c59 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:37.125583 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1116/2222] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 09:00:36 2022] POST /identity/v3/projects => generated 383 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:00:37.133493 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d32a929-af08-424e-8ce0-8af8203bb21d admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:37.197235 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:00:37.216300 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51933822-3a1c-485a-aa7d-effccebe1c59 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:37.223621 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d32a929-af08-424e-8ce0-8af8203bb21d admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:37.224332 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f241959-b982-40c7-a319-6a0a34bccb98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:37.225461 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f241959-b982-40c7-a319-6a0a34bccb98 admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:37.225935 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f241959-b982-40c7-a319-6a0a34bccb98 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:37.226423 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f241959-b982-40c7-a319-6a0a34bccb98 admin admin] PATH_INFO: `/v3/users` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:37.227462 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f241959-b982-40c7-a319-6a0a34bccb98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:37.229337 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e98d67d-4e55-4747-968f-0d19f143969d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:37.230808 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e98d67d-4e55-4747-968f-0d19f143969d admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:37.231484 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e98d67d-4e55-4747-968f-0d19f143969d admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:37.231767 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e98d67d-4e55-4747-968f-0d19f143969d admin admin] PATH_INFO: `/v3/users` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:37.231940 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f241959-b982-40c7-a319-6a0a34bccb98 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:37.232664 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e98d67d-4e55-4747-968f-0d19f143969d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:37.235059 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e98d67d-4e55-4747-968f-0d19f143969d admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:37.358121 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1117/2223] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 09:00:37 2022] POST /identity/v3/users => generated 292 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:00:37.362515 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e98d67d-4e55-4747-968f-0d19f143969d admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:37.376757 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e98d67d-4e55-4747-968f-0d19f143969d admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:37.386653 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1107/2224] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 09:00:37 2022] POST /identity/v3/users => generated 288 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:00:37.390283 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d365059-92f0-49e0-9826-6b7dad912b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:37.395753 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d365059-92f0-49e0-9826-6b7dad912b6f admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:37.395753 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d365059-92f0-49e0-9826-6b7dad912b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:37.395753 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d365059-92f0-49e0-9826-6b7dad912b6f admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:37.395753 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d365059-92f0-49e0-9826-6b7dad912b6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:37.396347 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f241959-b982-40c7-a319-6a0a34bccb98 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:37.396915 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d365059-92f0-49e0-9826-6b7dad912b6f admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:37.409785 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1118/2225] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:00:37 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 09:00:37.411175 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d365059-92f0-49e0-9826-6b7dad912b6f admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:37.413616 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f241959-b982-40c7-a319-6a0a34bccb98 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:37.421920 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f115b379-46d7-418f-9b9f-13b4c004d4ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:37.422970 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f115b379-46d7-418f-9b9f-13b4c004d4ee admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:37.423463 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f115b379-46d7-418f-9b9f-13b4c004d4ee admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:37.424028 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f115b379-46d7-418f-9b9f-13b4c004d4ee admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:37.425316 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f115b379-46d7-418f-9b9f-13b4c004d4ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:37.428453 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f115b379-46d7-418f-9b9f-13b4c004d4ee admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:37.429321 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d365059-92f0-49e0-9826-6b7dad912b6f admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:37.437566 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a25ab5-f45a-48d0-a5ae-c4aa79efaf86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:37.438568 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a25ab5-f45a-48d0-a5ae-c4aa79efaf86 admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:37.439064 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a25ab5-f45a-48d0-a5ae-c4aa79efaf86 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:37.439560 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a25ab5-f45a-48d0-a5ae-c4aa79efaf86 admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:37.440819 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a25ab5-f45a-48d0-a5ae-c4aa79efaf86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:37.443925 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a25ab5-f45a-48d0-a5ae-c4aa79efaf86 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:37.450225 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1108/2226] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:00:37 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:00:37.452967 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f115b379-46d7-418f-9b9f-13b4c004d4ee admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:37.461324 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1119/2227] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:00:37 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 09:00:37.466373 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a25ab5-f45a-48d0-a5ae-c4aa79efaf86 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:37.471812 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f115b379-46d7-418f-9b9f-13b4c004d4ee admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:37.477122 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a25ab5-f45a-48d0-a5ae-c4aa79efaf86 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:37.480410 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26417c24-5ed6-4dcb-8484-e7edc9b08ba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:37.481782 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26417c24-5ed6-4dcb-8484-e7edc9b08ba9 admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:37.482189 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6d85d2-79fd-4567-ac1e-3c49083f2fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:37.482313 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26417c24-5ed6-4dcb-8484-e7edc9b08ba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:37.482756 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26417c24-5ed6-4dcb-8484-e7edc9b08ba9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:37.482867 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6d85d2-79fd-4567-ac1e-3c49083f2fd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:37.483131 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6d85d2-79fd-4567-ac1e-3c49083f2fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:37.483401 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6d85d2-79fd-4567-ac1e-3c49083f2fd5 admin admin] PATH_INFO: `/v3/projects/eccd422b1e434b56ba0e0a31aebebd98/users/095ae2b97081460da5d3da2366a604c3/roles/b674fd0a592e4e0cada811bba5ceea4d` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:37.483930 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26417c24-5ed6-4dcb-8484-e7edc9b08ba9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:37.487132 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26417c24-5ed6-4dcb-8484-e7edc9b08ba9 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:37.499914 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6d85d2-79fd-4567-ac1e-3c49083f2fd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eccd422b1e434b56ba0e0a31aebebd98, user_id=095ae2b97081460da5d3da2366a604c3, role_id=b674fd0a592e4e0cada811bba5ceea4d)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:37.502649 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6d85d2-79fd-4567-ac1e-3c49083f2fd5 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:37.508937 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1109/2228] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:00:37 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 09:00:37.513814 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26417c24-5ed6-4dcb-8484-e7edc9b08ba9 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:37.524622 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26417c24-5ed6-4dcb-8484-e7edc9b08ba9 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:37.530038 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2508a2db-95aa-4059-a4fd-73fd4bfeab1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:37.531095 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2508a2db-95aa-4059-a4fd-73fd4bfeab1c admin admin] REQUEST_METHOD: `PUT` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:37.531484 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2508a2db-95aa-4059-a4fd-73fd4bfeab1c admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:37.531848 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2508a2db-95aa-4059-a4fd-73fd4bfeab1c admin admin] PATH_INFO: `/v3/projects/ee68d39628c44b38963d47685c611f05/users/151f80db2db745388c5f801d4092d28d/roles/b674fd0a592e4e0cada811bba5ceea4d` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:37.554181 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2508a2db-95aa-4059-a4fd-73fd4bfeab1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee68d39628c44b38963d47685c611f05, user_id=151f80db2db745388c5f801d4092d28d, role_id=b674fd0a592e4e0cada811bba5ceea4d)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:37.560809 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2508a2db-95aa-4059-a4fd-73fd4bfeab1c admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:37.591796 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1120/2229] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 09:00:37 2022] PUT /identity/v3/projects/eccd422b1e434b56ba0e0a31aebebd98/users/095ae2b97081460da5d3da2366a604c3/roles/b674fd0a592e4e0cada811bba5ceea4d => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:00:37.601294 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bd1226-abba-465c-9735-329bc62cff8f None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:37.601597 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bd1226-abba-465c-9735-329bc62cff8f None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:37.602144 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bd1226-abba-465c-9735-329bc62cff8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:37.633490 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-69bd1226-abba-465c-9735-329bc62cff8f None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:00:37.636011 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-69bd1226-abba-465c-9735-329bc62cff8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:37.694926 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1110/2230] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 09:00:37 2022] PUT /identity/v3/projects/ee68d39628c44b38963d47685c611f05/users/151f80db2db745388c5f801d4092d28d/roles/b674fd0a592e4e0cada811bba5ceea4d => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:00:37.706710 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a12ee51-900a-4d0c-a22a-3569e0b08a14 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:37.707726 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a12ee51-900a-4d0c-a22a-3569e0b08a14 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:37.708433 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a12ee51-900a-4d0c-a22a-3569e0b08a14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:37.738458 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:00:37.740374 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1121/2231] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:00:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:00:37.754392 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bd1226-abba-465c-9735-329bc62cff8f None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:37.756612 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-69bd1226-abba-465c-9735-329bc62cff8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:37.768089 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-2a12ee51-900a-4d0c-a22a-3569e0b08a14 None None] MFA Rules not processed for user `151f80db2db745388c5f801d4092d28d`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:00:37.770278 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-2a12ee51-900a-4d0c-a22a-3569e0b08a14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:37.886420 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bd1226-abba-465c-9735-329bc62cff8f None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:37.926187 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1111/2232] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:00:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:00:37.939979 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:00:37.943542 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a12ee51-900a-4d0c-a22a-3569e0b08a14 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:37.945779 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-2a12ee51-900a-4d0c-a22a-3569e0b08a14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:37.949957 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d502833a-2aeb-4612-9b66-284c76147a55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:37.950829 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d502833a-2aeb-4612-9b66-284c76147a55 service neutron] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:37.951262 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d502833a-2aeb-4612-9b66-284c76147a55 service neutron] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:37.951684 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d502833a-2aeb-4612-9b66-284c76147a55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:38.047079 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d502833a-2aeb-4612-9b66-284c76147a55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:38.050678 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a12ee51-900a-4d0c-a22a-3569e0b08a14 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:38.052599 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d502833a-2aeb-4612-9b66-284c76147a55 service neutron] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:38.054884 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81067c6f-9e57-4ab7-8f4f-aee490e70732 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbb9ac667f04c35a63b2689df18efed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:38.055587 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81067c6f-9e57-4ab7-8f4f-aee490e70732 service cinder] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:38.055938 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81067c6f-9e57-4ab7-8f4f-aee490e70732 service cinder] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:38.056412 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81067c6f-9e57-4ab7-8f4f-aee490e70732 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:38.066092 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81067c6f-9e57-4ab7-8f4f-aee490e70732 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:38.067787 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81067c6f-9e57-4ab7-8f4f-aee490e70732 service cinder] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:38.070571 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1122/2233] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:00:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 09:00:38.080171 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1112/2234] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:00:37 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:00:40.981557 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e111345d-de1d-41d1-8d4b-b7d7c70174bb None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:40.982052 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e111345d-de1d-41d1-8d4b-b7d7c70174bb None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:40.982175 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e111345d-de1d-41d1-8d4b-b7d7c70174bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:41.009734 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-e111345d-de1d-41d1-8d4b-b7d7c70174bb None None] MFA Rules not processed for user `f652a368b1cc42ae8b6f6c7a6dff6f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:00:41.011710 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-e111345d-de1d-41d1-8d4b-b7d7c70174bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:41.220864 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1123/2235] 10.222.0.55 () {58 vars in 1009 bytes} [Wed Aug 24 09:00:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:00:41.661551 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81067c6f-9e57-4ab7-8f4f-aee490e70732 service cinder] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:41.664444 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-81067c6f-9e57-4ab7-8f4f-aee490e70732 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:41.843187 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81067c6f-9e57-4ab7-8f4f-aee490e70732 service cinder] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:41.871183 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108c4f27-7c98-43e4-947d-cf2113241203 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0ebb5ccabef45378f0c1da69721e051', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:41.872610 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108c4f27-7c98-43e4-947d-cf2113241203 service swift] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:41.873269 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108c4f27-7c98-43e4-947d-cf2113241203 service swift] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:41.873761 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108c4f27-7c98-43e4-947d-cf2113241203 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:41.912321 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108c4f27-7c98-43e4-947d-cf2113241203 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:41.923505 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108c4f27-7c98-43e4-947d-cf2113241203 service swift] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:41.983553 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1113/2236] 10.222.0.55 () {60 vars in 1384 bytes} [Wed Aug 24 09:00:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 09:00:45.213609 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e111345d-de1d-41d1-8d4b-b7d7c70174bb None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:45.217939 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-e111345d-de1d-41d1-8d4b-b7d7c70174bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:45.408052 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e111345d-de1d-41d1-8d4b-b7d7c70174bb None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:45.415655 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5107a17-6951-41fd-a232-6bc9f98bb9c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17da357e555420799ee681bffbcc268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:45.416343 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5107a17-6951-41fd-a232-6bc9f98bb9c2 service nova] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:45.416658 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5107a17-6951-41fd-a232-6bc9f98bb9c2 service nova] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:45.416974 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5107a17-6951-41fd-a232-6bc9f98bb9c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:45.418842 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-b5107a17-6951-41fd-a232-6bc9f98bb9c2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:45.462557 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:00:45.543804 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5107a17-6951-41fd-a232-6bc9f98bb9c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:45.549394 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5107a17-6951-41fd-a232-6bc9f98bb9c2 service nova] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:45.581354 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1124/2237] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:00:45 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:00:45.702678 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108c4f27-7c98-43e4-947d-cf2113241203 service swift] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:45.704476 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-108c4f27-7c98-43e4-947d-cf2113241203 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:45.723353 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108c4f27-7c98-43e4-947d-cf2113241203 service swift] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:45.730540 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3608cc38-81b0-4d9d-acdb-75cb5cab047b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:45.731661 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3608cc38-81b0-4d9d-acdb-75cb5cab047b service neutron] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:45.732873 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3608cc38-81b0-4d9d-acdb-75cb5cab047b service neutron] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:45.733389 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3608cc38-81b0-4d9d-acdb-75cb5cab047b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:45.738352 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-3608cc38-81b0-4d9d-acdb-75cb5cab047b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:45.809064 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:00:45.933528 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3608cc38-81b0-4d9d-acdb-75cb5cab047b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:45.937853 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3608cc38-81b0-4d9d-acdb-75cb5cab047b service neutron] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:45.981522 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1114/2238] 10.222.0.55 () {60 vars in 1396 bytes} [Wed Aug 24 09:00:45 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3260 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:00:47.668526 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5107a17-6951-41fd-a232-6bc9f98bb9c2 service nova] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:47.702168 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5107a17-6951-41fd-a232-6bc9f98bb9c2 service nova] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:47.721242 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b470831d-27ca-4803-a873-1acbb0665d1a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17da357e555420799ee681bffbcc268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:47.732979 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b470831d-27ca-4803-a873-1acbb0665d1a service nova] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:47.735562 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b470831d-27ca-4803-a873-1acbb0665d1a service nova] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:47.735965 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b470831d-27ca-4803-a873-1acbb0665d1a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:47.740245 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-b470831d-27ca-4803-a873-1acbb0665d1a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:47.847467 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b470831d-27ca-4803-a873-1acbb0665d1a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:47.853428 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b470831d-27ca-4803-a873-1acbb0665d1a service nova] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:47.862289 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1125/2239] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:00:47 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:00:50.912069 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72006d65-c1bd-4100-b62d-12dab672c9d4 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:50.912566 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72006d65-c1bd-4100-b62d-12dab672c9d4 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:50.912566 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72006d65-c1bd-4100-b62d-12dab672c9d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:50.954853 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-72006d65-c1bd-4100-b62d-12dab672c9d4 None None] MFA Rules not processed for user `095ae2b97081460da5d3da2366a604c3`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:00:50.956776 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-72006d65-c1bd-4100-b62d-12dab672c9d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:51.099393 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1115/2240] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:00:50 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:00:51.099572 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b470831d-27ca-4803-a873-1acbb0665d1a service nova] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:51.130112 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b470831d-27ca-4803-a873-1acbb0665d1a service nova] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:51.136376 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7625c1bb-b33f-41c0-bee9-31ac3f0f7b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:51.137043 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7625c1bb-b33f-41c0-bee9-31ac3f0f7b12 admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:51.137324 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7625c1bb-b33f-41c0-bee9-31ac3f0f7b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:51.137774 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7625c1bb-b33f-41c0-bee9-31ac3f0f7b12 admin admin] PATH_INFO: `/v3/projects` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:51.138845 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7625c1bb-b33f-41c0-bee9-31ac3f0f7b12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:51.141901 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7625c1bb-b33f-41c0-bee9-31ac3f0f7b12 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:51.363139 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1126/2241] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 09:00:51 2022] POST /identity/v3/projects => generated 385 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:00:51.368443 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72006d65-c1bd-4100-b62d-12dab672c9d4 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:51.392202 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72006d65-c1bd-4100-b62d-12dab672c9d4 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:51.569609 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fa84e7-9da2-41f6-86d7-308d3d504fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:51.571353 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fa84e7-9da2-41f6-86d7-308d3d504fe7 admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:51.571742 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fa84e7-9da2-41f6-86d7-308d3d504fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:51.572115 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fa84e7-9da2-41f6-86d7-308d3d504fe7 admin admin] PATH_INFO: `/v3/users` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:51.572999 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fa84e7-9da2-41f6-86d7-308d3d504fe7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:51.591880 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fa84e7-9da2-41f6-86d7-308d3d504fe7 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:51.730549 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1116/2242] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 09:00:51 2022] POST /identity/v3/users => generated 299 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:00:51.732838 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7625c1bb-b33f-41c0-bee9-31ac3f0f7b12 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:51.755208 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7625c1bb-b33f-41c0-bee9-31ac3f0f7b12 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:51.767892 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755685bb-4287-44ec-9fdd-4d6fcf61ce09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:51.768988 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755685bb-4287-44ec-9fdd-4d6fcf61ce09 admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:51.772476 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755685bb-4287-44ec-9fdd-4d6fcf61ce09 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:51.774884 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755685bb-4287-44ec-9fdd-4d6fcf61ce09 admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:51.775816 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755685bb-4287-44ec-9fdd-4d6fcf61ce09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:51.787592 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755685bb-4287-44ec-9fdd-4d6fcf61ce09 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:51.814651 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1127/2243] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:00:51 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:00:51.820180 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fa84e7-9da2-41f6-86d7-308d3d504fe7 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:51.848290 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fa84e7-9da2-41f6-86d7-308d3d504fe7 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:51.870323 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfd32b3-4ab7-43bc-b2b7-fbf42b0c3627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:51.871029 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfd32b3-4ab7-43bc-b2b7-fbf42b0c3627 admin admin] REQUEST_METHOD: `PUT` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:51.871316 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfd32b3-4ab7-43bc-b2b7-fbf42b0c3627 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:51.872109 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfd32b3-4ab7-43bc-b2b7-fbf42b0c3627 admin admin] PATH_INFO: `/v3/projects/c2ee9cd323d14e18a73eaf58f8c71870/users/6eb4015e1c38446daeeeb884e5495870/roles/b870cd1acaa744e1ae1d58b9ac283821` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:51.931538 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfd32b3-4ab7-43bc-b2b7-fbf42b0c3627 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2ee9cd323d14e18a73eaf58f8c71870, user_id=6eb4015e1c38446daeeeb884e5495870, role_id=b870cd1acaa744e1ae1d58b9ac283821)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:51.934598 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfd32b3-4ab7-43bc-b2b7-fbf42b0c3627 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:52.084221 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1117/2244] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 09:00:51 2022] PUT /identity/v3/projects/c2ee9cd323d14e18a73eaf58f8c71870/users/6eb4015e1c38446daeeeb884e5495870/roles/b870cd1acaa744e1ae1d58b9ac283821 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:00:52.085120 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755685bb-4287-44ec-9fdd-4d6fcf61ce09 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:52.101228 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755685bb-4287-44ec-9fdd-4d6fcf61ce09 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:52.196674 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf498296-6116-4373-9c1e-d0d36e78f23a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:52.197316 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf498296-6116-4373-9c1e-d0d36e78f23a admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:52.200222 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf498296-6116-4373-9c1e-d0d36e78f23a admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:52.202018 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf498296-6116-4373-9c1e-d0d36e78f23a admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:52.202734 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf498296-6116-4373-9c1e-d0d36e78f23a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:52.204524 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf498296-6116-4373-9c1e-d0d36e78f23a admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:52.223221 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1128/2245] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:00:52 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:00:52.228515 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfd32b3-4ab7-43bc-b2b7-fbf42b0c3627 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:52.239857 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfd32b3-4ab7-43bc-b2b7-fbf42b0c3627 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:52.261050 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e98b69-08b2-4bfc-b8a0-10375065a9e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:52.261987 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e98b69-08b2-4bfc-b8a0-10375065a9e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:52.262516 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e98b69-08b2-4bfc-b8a0-10375065a9e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:52.262960 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e98b69-08b2-4bfc-b8a0-10375065a9e8 admin admin] PATH_INFO: `/v3/projects/c2ee9cd323d14e18a73eaf58f8c71870/users/6eb4015e1c38446daeeeb884e5495870/roles/b870cd1acaa744e1ae1d58b9ac283821` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:52.278733 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e98b69-08b2-4bfc-b8a0-10375065a9e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2ee9cd323d14e18a73eaf58f8c71870, user_id=6eb4015e1c38446daeeeb884e5495870, role_id=b870cd1acaa744e1ae1d58b9ac283821)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:52.281856 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e98b69-08b2-4bfc-b8a0-10375065a9e8 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:52.297147 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1118/2246] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 09:00:52 2022] PUT /identity/v3/projects/c2ee9cd323d14e18a73eaf58f8c71870/users/6eb4015e1c38446daeeeb884e5495870/roles/b870cd1acaa744e1ae1d58b9ac283821 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:00:57.061033 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf498296-6116-4373-9c1e-d0d36e78f23a admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:57.132584 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:00:57.162836 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf498296-6116-4373-9c1e-d0d36e78f23a admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:57.180329 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e98b69-08b2-4bfc-b8a0-10375065a9e8 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:57.204620 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:00:57.271775 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9038e68-e7d0-4d62-ad65-1877d04b55ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:57.272565 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9038e68-e7d0-4d62-ad65-1877d04b55ee service neutron] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:57.281737 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9038e68-e7d0-4d62-ad65-1877d04b55ee service neutron] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:57.281737 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9038e68-e7d0-4d62-ad65-1877d04b55ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:57.281737 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-a9038e68-e7d0-4d62-ad65-1877d04b55ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:57.305969 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9038e68-e7d0-4d62-ad65-1877d04b55ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:57.308135 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9038e68-e7d0-4d62-ad65-1877d04b55ee service neutron] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:57.330870 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1129/2247] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:00:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:00:57.362435 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e98b69-08b2-4bfc-b8a0-10375065a9e8 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:57.383736 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d84af2c-e4c5-4aaf-a0e5-e874e901e2cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:57.384646 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d84af2c-e4c5-4aaf-a0e5-e874e901e2cb service neutron] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:57.385102 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d84af2c-e4c5-4aaf-a0e5-e874e901e2cb service neutron] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:57.386038 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d84af2c-e4c5-4aaf-a0e5-e874e901e2cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:57.420205 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d84af2c-e4c5-4aaf-a0e5-e874e901e2cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:57.423501 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d84af2c-e4c5-4aaf-a0e5-e874e901e2cb service neutron] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:57.438549 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1119/2248] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:00:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:01:00.985233 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9038e68-e7d0-4d62-ad65-1877d04b55ee service neutron] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:01:01.076276 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9038e68-e7d0-4d62-ad65-1877d04b55ee service neutron] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:01:01.094689 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00dffb7-e3bb-4298-b91a-d4d29b97587a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:01:01.095606 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00dffb7-e3bb-4298-b91a-d4d29b97587a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:01:01.096515 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00dffb7-e3bb-4298-b91a-d4d29b97587a admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:01:01.096900 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00dffb7-e3bb-4298-b91a-d4d29b97587a admin admin] PATH_INFO: `/v3/users/151f80db2db745388c5f801d4092d28d` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:01:01.098342 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00dffb7-e3bb-4298-b91a-d4d29b97587a admin admin] RBAC: Authorizing `identity:delete_user(user_id=151f80db2db745388c5f801d4092d28d)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:01:01.100504 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00dffb7-e3bb-4298-b91a-d4d29b97587a admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:01:01.476509 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-b00dffb7-e3bb-4298-b91a-d4d29b97587a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '151f80db2db745388c5f801d4092d28d'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:01:01.477724 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-b00dffb7-e3bb-4298-b91a-d4d29b97587a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '151f80db2db745388c5f801d4092d28d'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:01:01.480149 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-b00dffb7-e3bb-4298-b91a-d4d29b97587a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '151f80db2db745388c5f801d4092d28d'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:01:01.517400 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-b00dffb7-e3bb-4298-b91a-d4d29b97587a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '151f80db2db745388c5f801d4092d28d'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:01:01.678786 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-b00dffb7-e3bb-4298-b91a-d4d29b97587a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '151f80db2db745388c5f801d4092d28d'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:01:01.713150 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1130/2249] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 09:01:00 2022] DELETE /identity/v3/users/151f80db2db745388c5f801d4092d28d => generated 0 bytes in 717 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:01:01.719812 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d739306c-77e8-4735-b5a0-84e33394c3fc None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:01:01.720632 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d739306c-77e8-4735-b5a0-84e33394c3fc None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:01:01.720632 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d739306c-77e8-4735-b5a0-84e33394c3fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:01:01.788871 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-d739306c-77e8-4735-b5a0-84e33394c3fc None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:01:01.793746 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-d739306c-77e8-4735-b5a0-84e33394c3fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:01:02.000525 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1120/2250] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:01:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:01:02.021437 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00dffb7-e3bb-4298-b91a-d4d29b97587a admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:01:02.023873 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-b00dffb7-e3bb-4298-b91a-d4d29b97587a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:01:02.071033 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:01:02.163483 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00dffb7-e3bb-4298-b91a-d4d29b97587a admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:01:02.172515 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf446542-d22f-4752-b3d1-6d86f1453403 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:01:02.173188 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf446542-d22f-4752-b3d1-6d86f1453403 service neutron] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:01:02.173467 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf446542-d22f-4752-b3d1-6d86f1453403 service neutron] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:01:02.173739 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf446542-d22f-4752-b3d1-6d86f1453403 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:01:02.214628 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf446542-d22f-4752-b3d1-6d86f1453403 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:01:02.216429 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf446542-d22f-4752-b3d1-6d86f1453403 service neutron] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:01:02.254155 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1131/2251] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:01:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:01:03.385722 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d739306c-77e8-4735-b5a0-84e33394c3fc None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:01:03.387260 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-d739306c-77e8-4735-b5a0-84e33394c3fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:01:03.429227 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d739306c-77e8-4735-b5a0-84e33394c3fc None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:01:03.437297 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11beee5a-83f6-469e-9dc0-d0106016b087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:01:03.438744 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11beee5a-83f6-469e-9dc0-d0106016b087 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:01:03.439675 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11beee5a-83f6-469e-9dc0-d0106016b087 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:01:03.440656 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11beee5a-83f6-469e-9dc0-d0106016b087 admin admin] PATH_INFO: `/v3/projects/ee68d39628c44b38963d47685c611f05` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:01:03.442807 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11beee5a-83f6-469e-9dc0-d0106016b087 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee68d39628c44b38963d47685c611f05)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:01:03.446645 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11beee5a-83f6-469e-9dc0-d0106016b087 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:01:03.653894 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-11beee5a-83f6-469e-9dc0-d0106016b087 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee68d39628c44b38963d47685c611f05', 'request_id': 'req-11beee5a-83f6-469e-9dc0-d0106016b087'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:01:03.754286 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-11beee5a-83f6-469e-9dc0-d0106016b087 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee68d39628c44b38963d47685c611f05', 'request_id': 'req-11beee5a-83f6-469e-9dc0-d0106016b087'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:01:03.768600 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-11beee5a-83f6-469e-9dc0-d0106016b087 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee68d39628c44b38963d47685c611f05', 'request_id': 'req-11beee5a-83f6-469e-9dc0-d0106016b087'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:01:03.793346 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-11beee5a-83f6-469e-9dc0-d0106016b087 admin admin] The token cache is being invalidate because project ee68d39628c44b38963d47685c611f05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:01:03.795729 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-11beee5a-83f6-469e-9dc0-d0106016b087 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:01:03.797529 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-11beee5a-83f6-469e-9dc0-d0106016b087 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:01:03.813336 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1121/2252] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 09:01:03 2022] DELETE /identity/v3/projects/ee68d39628c44b38963d47685c611f05 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:01:05.018321 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66f3a31-cbfe-476b-8fe5-a8d203845d07 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:01:05.019166 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66f3a31-cbfe-476b-8fe5-a8d203845d07 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:01:05.019636 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66f3a31-cbfe-476b-8fe5-a8d203845d07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:01:05.066717 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-c66f3a31-cbfe-476b-8fe5-a8d203845d07 None None] MFA Rules not processed for user `6eb4015e1c38446daeeeb884e5495870`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:01:05.068125 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-c66f3a31-cbfe-476b-8fe5-a8d203845d07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:01:05.210757 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1132/2253] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:01:05 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:01:05.228022 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11beee5a-83f6-469e-9dc0-d0106016b087 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:01:05.230266 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-11beee5a-83f6-469e-9dc0-d0106016b087 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:01:05.300322 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:01:05.327717 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11beee5a-83f6-469e-9dc0-d0106016b087 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:01:05.331809 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98a3b6e-9ea2-4717-afc0-e4044ae1e3a1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5512043ca9a7421b821741ca88d50b07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:01:05.332842 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98a3b6e-9ea2-4717-afc0-e4044ae1e3a1 service glance] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:01:05.333508 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98a3b6e-9ea2-4717-afc0-e4044ae1e3a1 service glance] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:01:05.333791 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98a3b6e-9ea2-4717-afc0-e4044ae1e3a1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:01:05.335797 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-d98a3b6e-9ea2-4717-afc0-e4044ae1e3a1 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:01:05.439417 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98a3b6e-9ea2-4717-afc0-e4044ae1e3a1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:01:05.442042 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98a3b6e-9ea2-4717-afc0-e4044ae1e3a1 service glance] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:01:05.455549 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1122/2254] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:01:05 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:01:05.887610 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad861ff0-bfe1-4529-857a-5e53dc384d79 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:01:05.888605 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad861ff0-bfe1-4529-857a-5e53dc384d79 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:01:05.888605 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad861ff0-bfe1-4529-857a-5e53dc384d79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:01:05.953504 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-ad861ff0-bfe1-4529-857a-5e53dc384d79 None None] MFA Rules not processed for user `f652a368b1cc42ae8b6f6c7a6dff6f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:01:05.962508 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-ad861ff0-bfe1-4529-857a-5e53dc384d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:01:06.079959 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1133/2255] 10.222.0.55 () {58 vars in 1009 bytes} [Wed Aug 24 09:01:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:01:06.126441 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98a3b6e-9ea2-4717-afc0-e4044ae1e3a1 service glance] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:01:06.128614 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-d98a3b6e-9ea2-4717-afc0-e4044ae1e3a1 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:01:06.257446 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98a3b6e-9ea2-4717-afc0-e4044ae1e3a1 service glance] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:01:06.263307 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd73b65-b40c-48fb-90b2-cf1ac635167d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0ebb5ccabef45378f0c1da69721e051', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:01:06.263943 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd73b65-b40c-48fb-90b2-cf1ac635167d service swift] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:01:06.264218 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd73b65-b40c-48fb-90b2-cf1ac635167d service swift] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:01:06.264487 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd73b65-b40c-48fb-90b2-cf1ac635167d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:01:06.275122 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd73b65-b40c-48fb-90b2-cf1ac635167d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:01:06.277704 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd73b65-b40c-48fb-90b2-cf1ac635167d service swift] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:01:06.289491 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1123/2256] 10.222.0.55 () {60 vars in 1384 bytes} [Wed Aug 24 09:01:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 09:01:09.267064 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad861ff0-bfe1-4529-857a-5e53dc384d79 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:01:09.269989 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-ad861ff0-bfe1-4529-857a-5e53dc384d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:01:09.353089 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:01:09.373709 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad861ff0-bfe1-4529-857a-5e53dc384d79 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:01:09.380398 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7d013a-b934-46c3-9c70-b85fc77f7ee3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:01:09.381281 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7d013a-b934-46c3-9c70-b85fc77f7ee3 service neutron] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:01:09.381607 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7d013a-b934-46c3-9c70-b85fc77f7ee3 service neutron] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:01:09.381886 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7d013a-b934-46c3-9c70-b85fc77f7ee3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:01:09.383821 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-be7d013a-b934-46c3-9c70-b85fc77f7ee3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:01:09.506523 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7d013a-b934-46c3-9c70-b85fc77f7ee3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:01:09.509069 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7d013a-b934-46c3-9c70-b85fc77f7ee3 service neutron] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:01:09.521251 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1134/2257] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:01:09 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:01:13.422905 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd73b65-b40c-48fb-90b2-cf1ac635167d service swift] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:01:13.424779 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-1cd73b65-b40c-48fb-90b2-cf1ac635167d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:01:13.568938 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd73b65-b40c-48fb-90b2-cf1ac635167d service swift] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:01:13.578415 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdaea05-d467-4f16-9187-7019a74090d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17da357e555420799ee681bffbcc268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:01:13.579603 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdaea05-d467-4f16-9187-7019a74090d8 service nova] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:01:13.580195 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdaea05-d467-4f16-9187-7019a74090d8 service nova] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:01:13.580663 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdaea05-d467-4f16-9187-7019a74090d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:01:13.584064 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-1bdaea05-d467-4f16-9187-7019a74090d8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:01:13.603389 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdaea05-d467-4f16-9187-7019a74090d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:01:13.605388 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdaea05-d467-4f16-9187-7019a74090d8 service nova] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:01:13.625582 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1124/2258] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:01:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 09:01:18.314589 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7d013a-b934-46c3-9c70-b85fc77f7ee3 service neutron] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:01:18.316292 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-be7d013a-b934-46c3-9c70-b85fc77f7ee3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:01:18.343345 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7d013a-b934-46c3-9c70-b85fc77f7ee3 service neutron] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:01:18.349009 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84d6807-4018-4f68-8213-ac067d30334c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:01:18.350836 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84d6807-4018-4f68-8213-ac067d30334c service neutron] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:01:18.351353 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84d6807-4018-4f68-8213-ac067d30334c service neutron] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:01:18.353027 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84d6807-4018-4f68-8213-ac067d30334c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:01:18.355578 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-c84d6807-4018-4f68-8213-ac067d30334c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:01:18.380482 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84d6807-4018-4f68-8213-ac067d30334c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:01:18.382186 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84d6807-4018-4f68-8213-ac067d30334c service neutron] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:01:18.398530 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1135/2259] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:01:18 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:01:18.470486 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdaea05-d467-4f16-9187-7019a74090d8 service nova] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:01:18.488194 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:01:18.496983 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdaea05-d467-4f16-9187-7019a74090d8 service nova] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:01:18.508229 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47208be8-f8be-4eac-ac9a-8876221b8eb3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17da357e555420799ee681bffbcc268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:01:18.509438 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47208be8-f8be-4eac-ac9a-8876221b8eb3 service nova] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:01:18.510224 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47208be8-f8be-4eac-ac9a-8876221b8eb3 service nova] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:01:18.510945 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47208be8-f8be-4eac-ac9a-8876221b8eb3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:01:18.516265 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-47208be8-f8be-4eac-ac9a-8876221b8eb3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:01:18.539019 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47208be8-f8be-4eac-ac9a-8876221b8eb3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:01:18.541659 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47208be8-f8be-4eac-ac9a-8876221b8eb3 service nova] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:01:18.558935 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1125/2260] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:01:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:01:19.707358 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451828f0-34c1-4e18-94ce-0f149353f2df None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:01:19.707358 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451828f0-34c1-4e18-94ce-0f149353f2df None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:01:19.707358 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451828f0-34c1-4e18-94ce-0f149353f2df None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:01:19.708332 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1136/2261] 10.222.0.55 () {54 vars in 921 bytes} [Wed Aug 24 09:01:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:01:19.718146 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c691fc-b94b-43e1-a6cb-0f0d82470066 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:01:19.719143 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c691fc-b94b-43e1-a6cb-0f0d82470066 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:01:19.719622 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c691fc-b94b-43e1-a6cb-0f0d82470066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:01:19.765174 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-d5c691fc-b94b-43e1-a6cb-0f0d82470066 None None] MFA Rules not processed for user `f17da357e555420799ee681bffbcc268`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:01:19.776317 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-d5c691fc-b94b-43e1-a6cb-0f0d82470066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:01:19.787661 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1126/2262] 10.222.0.55 () {58 vars in 1018 bytes} [Wed Aug 24 09:01:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:01:19.805989 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451828f0-34c1-4e18-94ce-0f149353f2df None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:01:19.823695 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451828f0-34c1-4e18-94ce-0f149353f2df None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:01:19.831034 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c6d0b3-d439-44d0-ac37-98652dec4258 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5512043ca9a7421b821741ca88d50b07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:01:19.832050 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c6d0b3-d439-44d0-ac37-98652dec4258 service glance] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:01:19.832567 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c6d0b3-d439-44d0-ac37-98652dec4258 service glance] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:01:19.833734 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c6d0b3-d439-44d0-ac37-98652dec4258 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:01:19.889181 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c6d0b3-d439-44d0-ac37-98652dec4258 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:01:19.891061 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c6d0b3-d439-44d0-ac37-98652dec4258 service glance] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:01:19.918213 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1137/2263] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:01:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:01:20.481238 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8360c0e-ca8d-4399-9895-247ffe76a68e None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:01:20.481650 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8360c0e-ca8d-4399-9895-247ffe76a68e None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:01:20.482055 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8360c0e-ca8d-4399-9895-247ffe76a68e None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:01:20.483654 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1127/2264] 10.222.0.55 () {54 vars in 921 bytes} [Wed Aug 24 09:01:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:01:20.489975 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a13032-6ef0-42f3-a6c2-e74bfcc7447d None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:01:20.490283 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a13032-6ef0-42f3-a6c2-e74bfcc7447d None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:01:20.490558 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a13032-6ef0-42f3-a6c2-e74bfcc7447d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:01:20.515913 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-84a13032-6ef0-42f3-a6c2-e74bfcc7447d None None] MFA Rules not processed for user `28b58388d1664510b0b507323bca5ae8`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:01:20.517197 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-84a13032-6ef0-42f3-a6c2-e74bfcc7447d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:01:20.533835 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1138/2265] 10.222.0.55 () {58 vars in 1018 bytes} [Wed Aug 24 09:01:20 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:01:20.548779 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8360c0e-ca8d-4399-9895-247ffe76a68e None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:01:20.561389 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8360c0e-ca8d-4399-9895-247ffe76a68e None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:01:20.567324 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5733f6f5-7056-4d98-9530-6459b2270ba9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:01:20.568348 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5733f6f5-7056-4d98-9530-6459b2270ba9 service neutron] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:01:20.568820 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5733f6f5-7056-4d98-9530-6459b2270ba9 service neutron] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:01:20.569331 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5733f6f5-7056-4d98-9530-6459b2270ba9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:01:20.582928 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5733f6f5-7056-4d98-9530-6459b2270ba9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:01:20.585680 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5733f6f5-7056-4d98-9530-6459b2270ba9 service neutron] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:01:20.601823 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1128/2266] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:01:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:01:20.701529 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057dd28c-cd24-4852-9c0f-3cc9e3aeebc9 None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:01:20.701891 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057dd28c-cd24-4852-9c0f-3cc9e3aeebc9 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:01:20.702173 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057dd28c-cd24-4852-9c0f-3cc9e3aeebc9 None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:01:20.703721 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1139/2267] 10.222.0.55 () {54 vars in 921 bytes} [Wed Aug 24 09:01:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:01:20.709894 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8207a4-3198-4ee7-b736-f55844be46cd None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:01:20.710823 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8207a4-3198-4ee7-b736-f55844be46cd None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:01:20.710823 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8207a4-3198-4ee7-b736-f55844be46cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:01:20.732428 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-db8207a4-3198-4ee7-b736-f55844be46cd None None] MFA Rules not processed for user `f17da357e555420799ee681bffbcc268`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:01:20.733702 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-db8207a4-3198-4ee7-b736-f55844be46cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:01:20.743553 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1129/2268] 10.222.0.55 () {58 vars in 1018 bytes} [Wed Aug 24 09:01:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:01:20.754884 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057dd28c-cd24-4852-9c0f-3cc9e3aeebc9 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:01:20.767256 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057dd28c-cd24-4852-9c0f-3cc9e3aeebc9 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:01:20.771170 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e556d2-3c46-4be1-880a-29c7914fa91a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:01:20.771788 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e556d2-3c46-4be1-880a-29c7914fa91a service neutron] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:01:20.772062 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e556d2-3c46-4be1-880a-29c7914fa91a service neutron] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:01:20.772332 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e556d2-3c46-4be1-880a-29c7914fa91a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:01:20.786948 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e556d2-3c46-4be1-880a-29c7914fa91a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:01:20.788311 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e556d2-3c46-4be1-880a-29c7914fa91a service neutron] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:01:20.802034 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1140/2269] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:01:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 09:01:20.890206 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8207a4-3198-4ee7-b736-f55844be46cd None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:01:20.892603 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-db8207a4-3198-4ee7-b736-f55844be46cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:01:21.012050 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8207a4-3198-4ee7-b736-f55844be46cd None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:01:21.024340 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f8b7b8-027d-4cad-971b-82d7f7a5eaf9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf0c99771661413aaa0f237af6b394ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:01:21.024340 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f8b7b8-027d-4cad-971b-82d7f7a5eaf9 service placement] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:01:21.024340 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f8b7b8-027d-4cad-971b-82d7f7a5eaf9 service placement] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:01:21.029264 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f8b7b8-027d-4cad-971b-82d7f7a5eaf9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:01:21.030440 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-f7f8b7b8-027d-4cad-971b-82d7f7a5eaf9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:01:21.040707 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e220c6-20fb-47be-879a-de4cd55ef212 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:01:21.041098 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e220c6-20fb-47be-879a-de4cd55ef212 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:01:21.041630 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e220c6-20fb-47be-879a-de4cd55ef212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:01:21.046177 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f8b7b8-027d-4cad-971b-82d7f7a5eaf9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:01:21.047161 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f8b7b8-027d-4cad-971b-82d7f7a5eaf9 service placement] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:01:21.060092 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1130/2270] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:01:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:01:21.086989 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-d5e220c6-20fb-47be-879a-de4cd55ef212 None None] MFA Rules not processed for user `095ae2b97081460da5d3da2366a604c3`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:01:21.090018 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-d5e220c6-20fb-47be-879a-de4cd55ef212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:01:21.104719 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1141/2271] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:01:21 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:01:25.903075 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d989bc-e08b-496e-a048-5402da78fea7 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:01:25.905940 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d989bc-e08b-496e-a048-5402da78fea7 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:01:25.905940 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d989bc-e08b-496e-a048-5402da78fea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:01:25.934846 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-00d989bc-e08b-496e-a048-5402da78fea7 None None] MFA Rules not processed for user `f652a368b1cc42ae8b6f6c7a6dff6f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:01:25.934846 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-00d989bc-e08b-496e-a048-5402da78fea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:01:25.942629 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1131/2272] 10.222.0.55 () {58 vars in 1009 bytes} [Wed Aug 24 09:01:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:01:25.958073 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e220c6-20fb-47be-879a-de4cd55ef212 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:01:25.980018 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e220c6-20fb-47be-879a-de4cd55ef212 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:01:25.987682 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad98a188-aa27-470a-ac78-880f8e73c79f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0ebb5ccabef45378f0c1da69721e051', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:01:25.988612 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad98a188-aa27-470a-ac78-880f8e73c79f service swift] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:01:25.989347 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad98a188-aa27-470a-ac78-880f8e73c79f service swift] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:01:25.989791 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad98a188-aa27-470a-ac78-880f8e73c79f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:01:26.002567 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad98a188-aa27-470a-ac78-880f8e73c79f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:01:26.004765 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad98a188-aa27-470a-ac78-880f8e73c79f service swift] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:01:26.026778 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1142/2273] 10.222.0.55 () {60 vars in 1384 bytes} [Wed Aug 24 09:01:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 09:01:29.168836 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d989bc-e08b-496e-a048-5402da78fea7 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:01:29.182185 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d989bc-e08b-496e-a048-5402da78fea7 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:01:29.204882 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5043958d-67a3-4db3-ae31-0db372c9db01 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf0c99771661413aaa0f237af6b394ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:01:29.205800 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5043958d-67a3-4db3-ae31-0db372c9db01 service placement] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:01:29.206373 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5043958d-67a3-4db3-ae31-0db372c9db01 service placement] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:01:29.206826 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5043958d-67a3-4db3-ae31-0db372c9db01 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:01:29.208776 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-5043958d-67a3-4db3-ae31-0db372c9db01 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:01:29.223348 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5043958d-67a3-4db3-ae31-0db372c9db01 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:01:29.225011 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5043958d-67a3-4db3-ae31-0db372c9db01 service placement] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:01:29.236578 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1132/2274] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:01:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 09:01:29.338026 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad98a188-aa27-470a-ac78-880f8e73c79f service swift] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:01:29.338026 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-ad98a188-aa27-470a-ac78-880f8e73c79f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:01:29.431908 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad98a188-aa27-470a-ac78-880f8e73c79f service swift] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:01:29.438836 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745a78ba-d707-4305-8ada-7cbb77a67bf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:01:29.439570 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745a78ba-d707-4305-8ada-7cbb77a67bf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:01:29.439849 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745a78ba-d707-4305-8ada-7cbb77a67bf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:01:29.440122 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745a78ba-d707-4305-8ada-7cbb77a67bf4 admin admin] PATH_INFO: `/v3/users/c11f1ac1772b46d2a03165571edb6f10` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:01:29.441491 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745a78ba-d707-4305-8ada-7cbb77a67bf4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c11f1ac1772b46d2a03165571edb6f10)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:01:29.443596 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745a78ba-d707-4305-8ada-7cbb77a67bf4 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:01:29.739901 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:01:29.762047 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-745a78ba-d707-4305-8ada-7cbb77a67bf4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c11f1ac1772b46d2a03165571edb6f10'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:01:29.762709 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-745a78ba-d707-4305-8ada-7cbb77a67bf4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c11f1ac1772b46d2a03165571edb6f10'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:01:29.763709 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-745a78ba-d707-4305-8ada-7cbb77a67bf4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c11f1ac1772b46d2a03165571edb6f10'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:01:29.778776 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-745a78ba-d707-4305-8ada-7cbb77a67bf4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c11f1ac1772b46d2a03165571edb6f10'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:01:29.894023 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-745a78ba-d707-4305-8ada-7cbb77a67bf4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c11f1ac1772b46d2a03165571edb6f10'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:01:29.908946 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1143/2275] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 09:01:29 2022] DELETE /identity/v3/users/c11f1ac1772b46d2a03165571edb6f10 => generated 0 bytes in 576 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:01:30.199350 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5043958d-67a3-4db3-ae31-0db372c9db01 service placement] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:01:30.227060 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5043958d-67a3-4db3-ae31-0db372c9db01 service placement] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:01:30.300888 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14001812-9883-47c4-a6ab-95269eaa0e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:01:30.301768 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14001812-9883-47c4-a6ab-95269eaa0e23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:01:30.302242 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14001812-9883-47c4-a6ab-95269eaa0e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:01:30.302862 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14001812-9883-47c4-a6ab-95269eaa0e23 admin admin] PATH_INFO: `/v3/projects/c1977f516dfe40f7b26ffae80d68edd9` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:01:30.304179 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14001812-9883-47c4-a6ab-95269eaa0e23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1977f516dfe40f7b26ffae80d68edd9)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:01:30.307120 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14001812-9883-47c4-a6ab-95269eaa0e23 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:01:30.480255 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:01:30.490852 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-14001812-9883-47c4-a6ab-95269eaa0e23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1977f516dfe40f7b26ffae80d68edd9', 'request_id': 'req-14001812-9883-47c4-a6ab-95269eaa0e23'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:01:30.613205 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-14001812-9883-47c4-a6ab-95269eaa0e23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1977f516dfe40f7b26ffae80d68edd9', 'request_id': 'req-14001812-9883-47c4-a6ab-95269eaa0e23'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:01:30.620039 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-14001812-9883-47c4-a6ab-95269eaa0e23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1977f516dfe40f7b26ffae80d68edd9', 'request_id': 'req-14001812-9883-47c4-a6ab-95269eaa0e23'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:01:30.632215 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-14001812-9883-47c4-a6ab-95269eaa0e23 admin admin] The token cache is being invalidate because project c1977f516dfe40f7b26ffae80d68edd9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:01:30.632510 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-14001812-9883-47c4-a6ab-95269eaa0e23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:01:30.633243 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-14001812-9883-47c4-a6ab-95269eaa0e23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:01:30.635272 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1133/2276] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 09:01:30 2022] DELETE /identity/v3/projects/c1977f516dfe40f7b26ffae80d68edd9 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:01:30.640289 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3d0df3-d92b-41d7-b466-9d842bc2dec0 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:01:30.640888 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3d0df3-d92b-41d7-b466-9d842bc2dec0 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:01:30.641008 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3d0df3-d92b-41d7-b466-9d842bc2dec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:01:30.664036 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-aa3d0df3-d92b-41d7-b466-9d842bc2dec0 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:01:30.665324 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-aa3d0df3-d92b-41d7-b466-9d842bc2dec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:01:30.727838 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1144/2277] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:01:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:01:30.875778 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af728d64-004a-4789-94a0-c66096110d05 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:01:30.875778 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af728d64-004a-4789-94a0-c66096110d05 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:01:30.875778 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af728d64-004a-4789-94a0-c66096110d05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:01:30.875778 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-af728d64-004a-4789-94a0-c66096110d05 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:01:30.875778 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-af728d64-004a-4789-94a0-c66096110d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:01:30.875778 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1134/2278] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:01:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:01:30.877904 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3d0df3-d92b-41d7-b466-9d842bc2dec0 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:01:30.877904 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3d0df3-d92b-41d7-b466-9d842bc2dec0 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:01:30.877904 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b480e8bb-cba5-434c-819a-96fa04e50cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:01:30.877904 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b480e8bb-cba5-434c-819a-96fa04e50cae admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:01:30.878610 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b480e8bb-cba5-434c-819a-96fa04e50cae admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:01:30.878610 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b480e8bb-cba5-434c-819a-96fa04e50cae admin admin] PATH_INFO: `/v3/domains` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:01:30.878610 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b480e8bb-cba5-434c-819a-96fa04e50cae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:01:30.878610 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b480e8bb-cba5-434c-819a-96fa04e50cae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:01:30.878610 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b480e8bb-cba5-434c-819a-96fa04e50cae admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:01:30.878610 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1145/2279] 10.222.0.55 () {58 vars in 1153 bytes} [Wed Aug 24 09:01:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:01:30.879269 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af728d64-004a-4789-94a0-c66096110d05 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:01:30.879269 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af728d64-004a-4789-94a0-c66096110d05 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:01:30.879269 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75fc431-0a34-4230-9af7-c0ebb9e8caca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:01:30.879269 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75fc431-0a34-4230-9af7-c0ebb9e8caca admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:01:30.879637 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75fc431-0a34-4230-9af7-c0ebb9e8caca admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:01:30.879637 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75fc431-0a34-4230-9af7-c0ebb9e8caca admin admin] PATH_INFO: `/v3/projects` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:01:30.879637 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75fc431-0a34-4230-9af7-c0ebb9e8caca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:01:30.879637 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75fc431-0a34-4230-9af7-c0ebb9e8caca admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:01:31.553692 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1135/2280] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 09:01:30 2022] POST /identity/v3/projects => generated 391 bytes in 718 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:01:31.555022 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b480e8bb-cba5-434c-819a-96fa04e50cae admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:01:31.566653 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b480e8bb-cba5-434c-819a-96fa04e50cae admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:01:31.632601 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc3a36c-59c8-4f54-be56-40d926a9b952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:01:31.633300 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc3a36c-59c8-4f54-be56-40d926a9b952 admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:01:31.633577 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc3a36c-59c8-4f54-be56-40d926a9b952 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:01:31.633911 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc3a36c-59c8-4f54-be56-40d926a9b952 admin admin] PATH_INFO: `/v3/users` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:01:31.634555 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc3a36c-59c8-4f54-be56-40d926a9b952 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:01:31.636502 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc3a36c-59c8-4f54-be56-40d926a9b952 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:01:31.722749 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1146/2281] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 09:01:31 2022] POST /identity/v3/users => generated 296 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:01:31.726826 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75fc431-0a34-4230-9af7-c0ebb9e8caca admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:01:31.738368 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75fc431-0a34-4230-9af7-c0ebb9e8caca admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:01:31.743178 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106a3a4f-5c9e-4345-aabd-9a81bccc922c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:01:31.743669 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106a3a4f-5c9e-4345-aabd-9a81bccc922c admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:01:31.743948 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106a3a4f-5c9e-4345-aabd-9a81bccc922c admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:01:31.744220 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106a3a4f-5c9e-4345-aabd-9a81bccc922c admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:01:31.744972 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106a3a4f-5c9e-4345-aabd-9a81bccc922c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:01:31.746553 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106a3a4f-5c9e-4345-aabd-9a81bccc922c admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:01:31.757653 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1136/2282] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:01:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:01:31.761611 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc3a36c-59c8-4f54-be56-40d926a9b952 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:01:31.772466 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc3a36c-59c8-4f54-be56-40d926a9b952 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:01:31.787581 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bff0940-1e15-466b-94dd-a84c30df60d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:01:31.787581 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bff0940-1e15-466b-94dd-a84c30df60d0 admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:01:31.787581 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bff0940-1e15-466b-94dd-a84c30df60d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:01:31.787581 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bff0940-1e15-466b-94dd-a84c30df60d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:01:31.787977 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bff0940-1e15-466b-94dd-a84c30df60d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:01:31.787977 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bff0940-1e15-466b-94dd-a84c30df60d0 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:01:31.793115 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1147/2283] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:01:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 09:01:31.797415 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106a3a4f-5c9e-4345-aabd-9a81bccc922c admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:01:31.807893 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106a3a4f-5c9e-4345-aabd-9a81bccc922c admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:01:31.812441 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a85b45-b005-4182-9278-78ea0ef283f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:01:31.813135 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a85b45-b005-4182-9278-78ea0ef283f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:01:31.813411 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a85b45-b005-4182-9278-78ea0ef283f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:01:31.813683 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a85b45-b005-4182-9278-78ea0ef283f1 admin admin] PATH_INFO: `/v3/projects/7423138c56394d85a8e0dbc72b448a76/users/31b4f0b867dd4ed6a1613e636a1867d2/roles/b674fd0a592e4e0cada811bba5ceea4d` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:01:31.829677 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a85b45-b005-4182-9278-78ea0ef283f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7423138c56394d85a8e0dbc72b448a76, user_id=31b4f0b867dd4ed6a1613e636a1867d2, role_id=b674fd0a592e4e0cada811bba5ceea4d)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:01:31.832292 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a85b45-b005-4182-9278-78ea0ef283f1 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:01:32.189005 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1137/2284] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 09:01:31 2022] PUT /identity/v3/projects/7423138c56394d85a8e0dbc72b448a76/users/31b4f0b867dd4ed6a1613e636a1867d2/roles/b674fd0a592e4e0cada811bba5ceea4d => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:01:32.200338 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b800d084-b056-4c18-999e-fe742c7dab4a None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:01:32.200580 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b800d084-b056-4c18-999e-fe742c7dab4a None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:01:32.200867 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b800d084-b056-4c18-999e-fe742c7dab4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:01:32.231382 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-b800d084-b056-4c18-999e-fe742c7dab4a None None] MFA Rules not processed for user `31b4f0b867dd4ed6a1613e636a1867d2`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:01:32.232699 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-b800d084-b056-4c18-999e-fe742c7dab4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:01:32.236813 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:01:32.301148 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1148/2285] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:01:32 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:01:32.311977 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a85b45-b005-4182-9278-78ea0ef283f1 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:01:32.314603 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-a7a85b45-b005-4182-9278-78ea0ef283f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:01:32.374507 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a85b45-b005-4182-9278-78ea0ef283f1 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:01:32.378403 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908521a4-6750-4c46-8eb3-40fcb6487de1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbb9ac667f04c35a63b2689df18efed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:01:32.379009 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908521a4-6750-4c46-8eb3-40fcb6487de1 service cinder] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:01:32.379282 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908521a4-6750-4c46-8eb3-40fcb6487de1 service cinder] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:01:32.379547 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908521a4-6750-4c46-8eb3-40fcb6487de1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:01:32.389806 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908521a4-6750-4c46-8eb3-40fcb6487de1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:01:32.391432 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908521a4-6750-4c46-8eb3-40fcb6487de1 service cinder] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:01:32.409076 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1138/2286] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:01:32 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:01:38.084829 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b800d084-b056-4c18-999e-fe742c7dab4a None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:01:38.086319 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-b800d084-b056-4c18-999e-fe742c7dab4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:01:38.175823 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b800d084-b056-4c18-999e-fe742c7dab4a None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:01:38.183339 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fdbe60-c26d-4f80-a418-7b3b36fc246b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17da357e555420799ee681bffbcc268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:01:38.183951 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fdbe60-c26d-4f80-a418-7b3b36fc246b service nova] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:01:38.184234 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fdbe60-c26d-4f80-a418-7b3b36fc246b service nova] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:01:38.184511 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fdbe60-c26d-4f80-a418-7b3b36fc246b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:01:38.186610 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-e9fdbe60-c26d-4f80-a418-7b3b36fc246b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:01:38.264382 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fdbe60-c26d-4f80-a418-7b3b36fc246b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:01:38.264382 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fdbe60-c26d-4f80-a418-7b3b36fc246b service nova] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:01:38.269878 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1149/2287] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:01:38 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:01:42.398017 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=64,keystone:DELETE=4,keystone:INSERT=3 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:01:43.558151 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8298bd92-cc3f-4f49-b405-594978008ea6 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:01:43.559552 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8298bd92-cc3f-4f49-b405-594978008ea6 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:01:43.560063 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8298bd92-cc3f-4f49-b405-594978008ea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:01:43.608981 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-8298bd92-cc3f-4f49-b405-594978008ea6 None None] MFA Rules not processed for user `f652a368b1cc42ae8b6f6c7a6dff6f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:01:43.612350 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-8298bd92-cc3f-4f49-b405-594978008ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:01:43.757833 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1139/2288] 10.222.0.55 () {58 vars in 1009 bytes} [Wed Aug 24 09:01:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:01:43.788678 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fdbe60-c26d-4f80-a418-7b3b36fc246b service nova] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:01:43.789292 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-e9fdbe60-c26d-4f80-a418-7b3b36fc246b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:01:43.851265 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:01:43.876892 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fdbe60-c26d-4f80-a418-7b3b36fc246b service nova] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:01:43.882357 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f522b32-313f-46a5-bf88-bc9cef510c6a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0ebb5ccabef45378f0c1da69721e051', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:01:43.883018 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f522b32-313f-46a5-bf88-bc9cef510c6a service swift] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:01:43.883870 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f522b32-313f-46a5-bf88-bc9cef510c6a service swift] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:01:43.884154 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f522b32-313f-46a5-bf88-bc9cef510c6a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:01:43.899893 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f522b32-313f-46a5-bf88-bc9cef510c6a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:01:43.904379 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f522b32-313f-46a5-bf88-bc9cef510c6a service swift] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:01:43.914314 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1150/2289] 10.222.0.55 () {60 vars in 1384 bytes} [Wed Aug 24 09:01:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 09:01:53.753324 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=25 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:01:54.047783 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=13 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:02:07.653916 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8298bd92-cc3f-4f49-b405-594978008ea6 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:02:07.718680 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8298bd92-cc3f-4f49-b405-594978008ea6 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:02:07.723326 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4194605b-54cb-421d-aa0d-ade821f9ad37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:02:07.723954 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4194605b-54cb-421d-aa0d-ade821f9ad37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:02:07.724233 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4194605b-54cb-421d-aa0d-ade821f9ad37 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:02:07.724506 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4194605b-54cb-421d-aa0d-ade821f9ad37 admin admin] PATH_INFO: `/v3/users/31b4f0b867dd4ed6a1613e636a1867d2` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:02:07.725657 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4194605b-54cb-421d-aa0d-ade821f9ad37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=31b4f0b867dd4ed6a1613e636a1867d2)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:02:07.728252 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4194605b-54cb-421d-aa0d-ade821f9ad37 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:02:08.024293 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-4194605b-54cb-421d-aa0d-ade821f9ad37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31b4f0b867dd4ed6a1613e636a1867d2'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:02:08.025055 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-4194605b-54cb-421d-aa0d-ade821f9ad37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31b4f0b867dd4ed6a1613e636a1867d2'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:02:08.025671 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-4194605b-54cb-421d-aa0d-ade821f9ad37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31b4f0b867dd4ed6a1613e636a1867d2'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:02:08.044709 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-4194605b-54cb-421d-aa0d-ade821f9ad37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31b4f0b867dd4ed6a1613e636a1867d2'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:02:08.155828 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-4194605b-54cb-421d-aa0d-ade821f9ad37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31b4f0b867dd4ed6a1613e636a1867d2'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:02:08.171200 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1140/2290] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 09:02:07 2022] DELETE /identity/v3/users/31b4f0b867dd4ed6a1613e636a1867d2 => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:02:08.174567 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c5f952-0717-44d5-928d-69720affa9f1 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:02:08.175303 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c5f952-0717-44d5-928d-69720affa9f1 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:02:08.175591 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c5f952-0717-44d5-928d-69720affa9f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:02:08.197500 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-c8c5f952-0717-44d5-928d-69720affa9f1 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:02:08.198733 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-c8c5f952-0717-44d5-928d-69720affa9f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:02:08.261710 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1151/2291] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:02:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:02:08.271755 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4194605b-54cb-421d-aa0d-ade821f9ad37 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:02:08.273858 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-4194605b-54cb-421d-aa0d-ade821f9ad37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:02:08.340209 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4194605b-54cb-421d-aa0d-ade821f9ad37 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:02:08.344466 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a1acc3-dcc5-49f3-a54c-52ad9d61cc49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:02:08.345301 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a1acc3-dcc5-49f3-a54c-52ad9d61cc49 service neutron] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:02:08.345623 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a1acc3-dcc5-49f3-a54c-52ad9d61cc49 service neutron] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:02:08.345987 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a1acc3-dcc5-49f3-a54c-52ad9d61cc49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:02:08.357334 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a1acc3-dcc5-49f3-a54c-52ad9d61cc49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:02:08.358929 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a1acc3-dcc5-49f3-a54c-52ad9d61cc49 service neutron] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:02:08.370371 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1141/2292] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:02:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:02:08.856484 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c5f952-0717-44d5-928d-69720affa9f1 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:02:08.856484 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-c8c5f952-0717-44d5-928d-69720affa9f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:02:08.856484 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c5f952-0717-44d5-928d-69720affa9f1 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:02:08.860101 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64728df6-35ba-494b-a154-03a302a07c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:02:08.863948 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64728df6-35ba-494b-a154-03a302a07c41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:02:08.864278 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64728df6-35ba-494b-a154-03a302a07c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:02:08.864577 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64728df6-35ba-494b-a154-03a302a07c41 admin admin] PATH_INFO: `/v3/projects/7423138c56394d85a8e0dbc72b448a76` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:02:08.866024 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64728df6-35ba-494b-a154-03a302a07c41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7423138c56394d85a8e0dbc72b448a76)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:02:08.869051 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64728df6-35ba-494b-a154-03a302a07c41 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:02:09.014559 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-64728df6-35ba-494b-a154-03a302a07c41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7423138c56394d85a8e0dbc72b448a76', 'request_id': 'req-64728df6-35ba-494b-a154-03a302a07c41'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:02:09.020359 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-64728df6-35ba-494b-a154-03a302a07c41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7423138c56394d85a8e0dbc72b448a76', 'request_id': 'req-64728df6-35ba-494b-a154-03a302a07c41'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:02:09.092845 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-64728df6-35ba-494b-a154-03a302a07c41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7423138c56394d85a8e0dbc72b448a76', 'request_id': 'req-64728df6-35ba-494b-a154-03a302a07c41'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:02:09.107662 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-64728df6-35ba-494b-a154-03a302a07c41 admin admin] The token cache is being invalidate because project 7423138c56394d85a8e0dbc72b448a76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:02:09.108369 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-64728df6-35ba-494b-a154-03a302a07c41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:02:09.109500 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-64728df6-35ba-494b-a154-03a302a07c41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:02:09.111178 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1152/2293] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 09:02:08 2022] DELETE /identity/v3/projects/7423138c56394d85a8e0dbc72b448a76 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:02:09.428757 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a1acc3-dcc5-49f3-a54c-52ad9d61cc49 service neutron] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:02:09.430276 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-a2a1acc3-dcc5-49f3-a54c-52ad9d61cc49 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:02:09.457171 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:02:09.505730 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a1acc3-dcc5-49f3-a54c-52ad9d61cc49 service neutron] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:02:09.577778 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058b0916-8d65-4c4a-84e2-9ad8781bfbf8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17da357e555420799ee681bffbcc268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:02:09.578390 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b0916-8d65-4c4a-84e2-9ad8781bfbf8 service nova] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:02:09.578669 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b0916-8d65-4c4a-84e2-9ad8781bfbf8 service nova] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:02:09.578956 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b0916-8d65-4c4a-84e2-9ad8781bfbf8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:02:09.580952 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-058b0916-8d65-4c4a-84e2-9ad8781bfbf8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:02:09.594251 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058b0916-8d65-4c4a-84e2-9ad8781bfbf8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:02:09.596178 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058b0916-8d65-4c4a-84e2-9ad8781bfbf8 service nova] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:02:09.607489 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1142/2294] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:02:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:02:19.103283 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:02:19.789965 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=52 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:02:21.127564 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7fcf83-ce9d-4d1b-8f3c-3687ac6b2db1 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:02:21.128074 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7fcf83-ce9d-4d1b-8f3c-3687ac6b2db1 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:02:21.128207 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7fcf83-ce9d-4d1b-8f3c-3687ac6b2db1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:02:21.151216 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-2f7fcf83-ce9d-4d1b-8f3c-3687ac6b2db1 None None] MFA Rules not processed for user `f652a368b1cc42ae8b6f6c7a6dff6f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:02:21.152475 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-2f7fcf83-ce9d-4d1b-8f3c-3687ac6b2db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:02:21.230238 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1153/2295] 10.222.0.55 () {58 vars in 967 bytes} [Wed Aug 24 09:02:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:02:21.246833 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058b0916-8d65-4c4a-84e2-9ad8781bfbf8 service nova] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:02:21.252827 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-058b0916-8d65-4c4a-84e2-9ad8781bfbf8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:02:21.331732 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058b0916-8d65-4c4a-84e2-9ad8781bfbf8 service nova] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:02:21.336994 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85a37ec-5090-4dae-81be-07d95997afb2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0ebb5ccabef45378f0c1da69721e051', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:02:21.337612 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85a37ec-5090-4dae-81be-07d95997afb2 service swift] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:02:21.337889 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85a37ec-5090-4dae-81be-07d95997afb2 service swift] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:02:21.338156 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85a37ec-5090-4dae-81be-07d95997afb2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:02:21.347273 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85a37ec-5090-4dae-81be-07d95997afb2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:02:21.348935 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85a37ec-5090-4dae-81be-07d95997afb2 service swift] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:02:21.357888 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1143/2296] 10.222.0.55 () {60 vars in 1384 bytes} [Wed Aug 24 09:02:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 09:02:28.908640 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7fcf83-ce9d-4d1b-8f3c-3687ac6b2db1 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:02:28.910095 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-2f7fcf83-ce9d-4d1b-8f3c-3687ac6b2db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:02:28.972168 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7fcf83-ce9d-4d1b-8f3c-3687ac6b2db1 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:02:28.976839 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c14400-bc86-44db-b95c-effab9d2ecc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:02:28.977519 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c14400-bc86-44db-b95c-effab9d2ecc9 service neutron] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:02:28.977730 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c14400-bc86-44db-b95c-effab9d2ecc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:02:28.978001 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c14400-bc86-44db-b95c-effab9d2ecc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:02:28.979968 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-36c14400-bc86-44db-b95c-effab9d2ecc9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:02:29.048099 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c14400-bc86-44db-b95c-effab9d2ecc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:02:29.050673 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c14400-bc86-44db-b95c-effab9d2ecc9 service neutron] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:02:29.064921 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1154/2297] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:02:28 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:02:31.354435 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=32 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:02:39.060005 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=75 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:02:44.222066 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7886d0-eb49-4c07-9bb1-e14526ac29e5 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:02:44.222609 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7886d0-eb49-4c07-9bb1-e14526ac29e5 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:02:44.222609 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7886d0-eb49-4c07-9bb1-e14526ac29e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:02:44.244754 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-fb7886d0-eb49-4c07-9bb1-e14526ac29e5 None None] MFA Rules not processed for user `f56524e822e641bc9bb11eddfe27f85d`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:02:44.245983 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-fb7886d0-eb49-4c07-9bb1-e14526ac29e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:02:44.309678 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1144/2298] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:02:44 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:02:54.301876 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=25 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:03:37.203560 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d6a386-483d-4a97-9727-f147441b3948 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:37.203560 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d6a386-483d-4a97-9727-f147441b3948 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:37.203560 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d6a386-483d-4a97-9727-f147441b3948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:37.233520 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-b8d6a386-483d-4a97-9727-f147441b3948 None None] MFA Rules not processed for user `993697d33b7841668b1eabcdcd6b40f9`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:03:37.234791 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-b8d6a386-483d-4a97-9727-f147441b3948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:03:37.319505 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1155/2299] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:03:37 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:03:37.642602 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7886d0-eb49-4c07-9bb1-e14526ac29e5 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:37.644678 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-fb7886d0-eb49-4c07-9bb1-e14526ac29e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:03:37.691156 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7886d0-eb49-4c07-9bb1-e14526ac29e5 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:37.694961 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bfd4bd-1624-4d96-81e5-1a59b3d6b2ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:37.695540 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bfd4bd-1624-4d96-81e5-1a59b3d6b2ef service neutron] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:37.695756 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bfd4bd-1624-4d96-81e5-1a59b3d6b2ef service neutron] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:37.696024 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bfd4bd-1624-4d96-81e5-1a59b3d6b2ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:37.697776 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-75bfd4bd-1624-4d96-81e5-1a59b3d6b2ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:03:37.707171 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bfd4bd-1624-4d96-81e5-1a59b3d6b2ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:37.708611 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bfd4bd-1624-4d96-81e5-1a59b3d6b2ef service neutron] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:37.720882 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1145/2300] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:03:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:03:47.317435 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=37 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:03:47.713477 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=19 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:03:55.476527 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f968c8ee-81de-41dd-9aa8-b0adb22a9e2e None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:55.477468 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f968c8ee-81de-41dd-9aa8-b0adb22a9e2e None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:55.477956 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f968c8ee-81de-41dd-9aa8-b0adb22a9e2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:55.510513 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-f968c8ee-81de-41dd-9aa8-b0adb22a9e2e None None] MFA Rules not processed for user `1105970982014891902ebbb9904a57a5`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:03:55.512803 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-f968c8ee-81de-41dd-9aa8-b0adb22a9e2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:03:55.625712 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1156/2301] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:03:55 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:04:03.714302 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bfd4bd-1624-4d96-81e5-1a59b3d6b2ef service neutron] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:03.715807 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-75bfd4bd-1624-4d96-81e5-1a59b3d6b2ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:03.829059 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bfd4bd-1624-4d96-81e5-1a59b3d6b2ef service neutron] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:03.834722 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65dde44-43f5-4f26-a871-7eedbe8c3bde service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf0c99771661413aaa0f237af6b394ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:03.835446 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65dde44-43f5-4f26-a871-7eedbe8c3bde service placement] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:03.835785 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65dde44-43f5-4f26-a871-7eedbe8c3bde service placement] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:03.836119 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65dde44-43f5-4f26-a871-7eedbe8c3bde service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:03.839672 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-c65dde44-43f5-4f26-a871-7eedbe8c3bde service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:03.853636 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65dde44-43f5-4f26-a871-7eedbe8c3bde service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:03.855269 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65dde44-43f5-4f26-a871-7eedbe8c3bde service placement] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:03.866247 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1146/2302] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:04:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:04:05.613738 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=25 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:04:13.861488 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=44 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:04:15.538833 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f968c8ee-81de-41dd-9aa8-b0adb22a9e2e None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:15.541825 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-f968c8ee-81de-41dd-9aa8-b0adb22a9e2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:15.616294 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f968c8ee-81de-41dd-9aa8-b0adb22a9e2e None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:15.625959 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9428964d-ba72-46a2-a4f8-30642c7e9a73 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17da357e555420799ee681bffbcc268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:15.626932 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9428964d-ba72-46a2-a4f8-30642c7e9a73 service nova] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:15.627375 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9428964d-ba72-46a2-a4f8-30642c7e9a73 service nova] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:15.627807 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9428964d-ba72-46a2-a4f8-30642c7e9a73 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:15.630915 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-9428964d-ba72-46a2-a4f8-30642c7e9a73 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:15.770560 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9428964d-ba72-46a2-a4f8-30642c7e9a73 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:15.772370 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9428964d-ba72-46a2-a4f8-30642c7e9a73 service nova] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:15.787242 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1157/2303] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:04:15 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:04:19.373972 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355a89b1-bd66-494b-aa6b-e1ceec4bcce7 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:19.380273 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355a89b1-bd66-494b-aa6b-e1ceec4bcce7 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:19.380273 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355a89b1-bd66-494b-aa6b-e1ceec4bcce7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:19.396785 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-355a89b1-bd66-494b-aa6b-e1ceec4bcce7 None None] MFA Rules not processed for user `c2c3752027f342a489e6eb9d3b540819`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:04:19.398059 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-355a89b1-bd66-494b-aa6b-e1ceec4bcce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:19.413124 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1147/2304] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:04:19 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:04:22.361805 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9428964d-ba72-46a2-a4f8-30642c7e9a73 service nova] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:22.391204 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9428964d-ba72-46a2-a4f8-30642c7e9a73 service nova] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:22.393481 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b1ee82-f6be-4d08-b790-5d04b9093d41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:22.394318 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b1ee82-f6be-4d08-b790-5d04b9093d41 service neutron] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:22.394596 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b1ee82-f6be-4d08-b790-5d04b9093d41 service neutron] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:22.394862 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b1ee82-f6be-4d08-b790-5d04b9093d41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:22.397242 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-80b1ee82-f6be-4d08-b790-5d04b9093d41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:22.526775 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b1ee82-f6be-4d08-b790-5d04b9093d41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:22.530310 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b1ee82-f6be-4d08-b790-5d04b9093d41 service neutron] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:22.554091 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1158/2305] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:04:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:04:28.284218 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355a89b1-bd66-494b-aa6b-e1ceec4bcce7 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:28.286262 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-355a89b1-bd66-494b-aa6b-e1ceec4bcce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:28.302315 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355a89b1-bd66-494b-aa6b-e1ceec4bcce7 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:28.309576 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448a0ee4-9dea-422d-886c-89bc5b50d8a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:28.310053 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448a0ee4-9dea-422d-886c-89bc5b50d8a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:28.310291 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448a0ee4-9dea-422d-886c-89bc5b50d8a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:28.310558 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448a0ee4-9dea-422d-886c-89bc5b50d8a2 admin admin] PATH_INFO: `/v3/users/f56524e822e641bc9bb11eddfe27f85d` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:28.311701 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448a0ee4-9dea-422d-886c-89bc5b50d8a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f56524e822e641bc9bb11eddfe27f85d)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:28.313716 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448a0ee4-9dea-422d-886c-89bc5b50d8a2 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:28.540603 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-448a0ee4-9dea-422d-886c-89bc5b50d8a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f56524e822e641bc9bb11eddfe27f85d'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:28.541292 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-448a0ee4-9dea-422d-886c-89bc5b50d8a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f56524e822e641bc9bb11eddfe27f85d'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:28.541912 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-448a0ee4-9dea-422d-886c-89bc5b50d8a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f56524e822e641bc9bb11eddfe27f85d'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:28.556387 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-448a0ee4-9dea-422d-886c-89bc5b50d8a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f56524e822e641bc9bb11eddfe27f85d'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:28.667623 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-448a0ee4-9dea-422d-886c-89bc5b50d8a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f56524e822e641bc9bb11eddfe27f85d'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:28.685368 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1148/2306] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 09:04:28 2022] DELETE /identity/v3/users/f56524e822e641bc9bb11eddfe27f85d => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:04:29.010315 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b1ee82-f6be-4d08-b790-5d04b9093d41 service neutron] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:29.022721 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b1ee82-f6be-4d08-b790-5d04b9093d41 service neutron] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:29.054758 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:04:29.087155 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76860b76-5abc-435b-981a-a3ba3ce58a85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:29.087830 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76860b76-5abc-435b-981a-a3ba3ce58a85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:29.088115 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76860b76-5abc-435b-981a-a3ba3ce58a85 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:29.089839 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76860b76-5abc-435b-981a-a3ba3ce58a85 admin admin] PATH_INFO: `/v3/projects/c966c27f100846ca8cf06122ddc8c408` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:29.091061 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76860b76-5abc-435b-981a-a3ba3ce58a85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c966c27f100846ca8cf06122ddc8c408)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:29.093114 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76860b76-5abc-435b-981a-a3ba3ce58a85 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:29.217569 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-76860b76-5abc-435b-981a-a3ba3ce58a85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c966c27f100846ca8cf06122ddc8c408', 'request_id': 'req-76860b76-5abc-435b-981a-a3ba3ce58a85'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:29.223645 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-76860b76-5abc-435b-981a-a3ba3ce58a85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c966c27f100846ca8cf06122ddc8c408', 'request_id': 'req-76860b76-5abc-435b-981a-a3ba3ce58a85'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:29.328997 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-76860b76-5abc-435b-981a-a3ba3ce58a85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c966c27f100846ca8cf06122ddc8c408', 'request_id': 'req-76860b76-5abc-435b-981a-a3ba3ce58a85'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:29.341282 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-76860b76-5abc-435b-981a-a3ba3ce58a85 admin admin] The token cache is being invalidate because project c966c27f100846ca8cf06122ddc8c408 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:04:29.341500 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-76860b76-5abc-435b-981a-a3ba3ce58a85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:29.342144 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-76860b76-5abc-435b-981a-a3ba3ce58a85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:29.343634 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1159/2307] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 09:04:29 2022] DELETE /identity/v3/projects/c966c27f100846ca8cf06122ddc8c408 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:04:29.348654 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c2ceef-3c75-4d4a-98c3-4aad7a3a88dc None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:29.349161 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c2ceef-3c75-4d4a-98c3-4aad7a3a88dc None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:29.349268 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c2ceef-3c75-4d4a-98c3-4aad7a3a88dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:29.412862 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-17c2ceef-3c75-4d4a-98c3-4aad7a3a88dc None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:04:29.414097 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-17c2ceef-3c75-4d4a-98c3-4aad7a3a88dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:29.480918 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1149/2308] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:04:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:04:29.489277 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb25a5fa-4a75-4787-ba3c-1887f9494af5 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:29.490150 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb25a5fa-4a75-4787-ba3c-1887f9494af5 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:29.490150 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb25a5fa-4a75-4787-ba3c-1887f9494af5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:29.512180 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-bb25a5fa-4a75-4787-ba3c-1887f9494af5 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:04:29.513557 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-bb25a5fa-4a75-4787-ba3c-1887f9494af5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:29.528390 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1160/2309] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:04:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:04:29.533417 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c2ceef-3c75-4d4a-98c3-4aad7a3a88dc None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:29.544764 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c2ceef-3c75-4d4a-98c3-4aad7a3a88dc None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:29.550121 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78d22cc-916f-4a61-bbb9-4a0288f297db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:29.567145 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78d22cc-916f-4a61-bbb9-4a0288f297db admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:29.567145 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78d22cc-916f-4a61-bbb9-4a0288f297db admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:29.567145 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78d22cc-916f-4a61-bbb9-4a0288f297db admin admin] PATH_INFO: `/v3/domains` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:29.567145 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78d22cc-916f-4a61-bbb9-4a0288f297db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:04:29.567145 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78d22cc-916f-4a61-bbb9-4a0288f297db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:29.567145 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78d22cc-916f-4a61-bbb9-4a0288f297db admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:29.568944 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1150/2310] 10.222.0.55 () {58 vars in 1153 bytes} [Wed Aug 24 09:04:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:04:29.574385 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb25a5fa-4a75-4787-ba3c-1887f9494af5 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:29.584288 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb25a5fa-4a75-4787-ba3c-1887f9494af5 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:29.590024 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ff153d-7744-4088-91c8-25888b8e4b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:29.590662 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ff153d-7744-4088-91c8-25888b8e4b2b admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:29.591133 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ff153d-7744-4088-91c8-25888b8e4b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:29.591546 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ff153d-7744-4088-91c8-25888b8e4b2b admin admin] PATH_INFO: `/v3/projects` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:29.592215 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ff153d-7744-4088-91c8-25888b8e4b2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:29.594947 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ff153d-7744-4088-91c8-25888b8e4b2b admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:29.749900 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1161/2311] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 09:04:29 2022] POST /identity/v3/projects => generated 357 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:04:29.754210 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78d22cc-916f-4a61-bbb9-4a0288f297db admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:29.764925 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78d22cc-916f-4a61-bbb9-4a0288f297db admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:29.785716 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:04:29.834723 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee215c1f-2123-499c-a5d1-88eee8a69b25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:29.835353 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee215c1f-2123-499c-a5d1-88eee8a69b25 admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:29.835666 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee215c1f-2123-499c-a5d1-88eee8a69b25 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:29.835975 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee215c1f-2123-499c-a5d1-88eee8a69b25 admin admin] PATH_INFO: `/v3/users` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:29.836799 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee215c1f-2123-499c-a5d1-88eee8a69b25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:29.839051 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee215c1f-2123-499c-a5d1-88eee8a69b25 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:29.986436 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1151/2312] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 09:04:29 2022] POST /identity/v3/users => generated 279 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:04:29.990599 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ff153d-7744-4088-91c8-25888b8e4b2b admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:30.001495 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ff153d-7744-4088-91c8-25888b8e4b2b admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:30.006307 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00b5f72-5d24-4efc-852c-3eda51db5f0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:30.006958 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00b5f72-5d24-4efc-852c-3eda51db5f0d admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:30.007192 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00b5f72-5d24-4efc-852c-3eda51db5f0d admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:30.007463 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00b5f72-5d24-4efc-852c-3eda51db5f0d admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:30.008212 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00b5f72-5d24-4efc-852c-3eda51db5f0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:30.009850 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00b5f72-5d24-4efc-852c-3eda51db5f0d admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:30.020870 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1162/2313] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:04:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:04:30.025425 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee215c1f-2123-499c-a5d1-88eee8a69b25 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:30.036390 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee215c1f-2123-499c-a5d1-88eee8a69b25 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:30.041956 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8773c3-4d33-402a-b437-6139853f4c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:30.043263 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8773c3-4d33-402a-b437-6139853f4c06 admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:30.043263 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8773c3-4d33-402a-b437-6139853f4c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:30.043471 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8773c3-4d33-402a-b437-6139853f4c06 admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:30.044667 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8773c3-4d33-402a-b437-6139853f4c06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:30.045916 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8773c3-4d33-402a-b437-6139853f4c06 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:30.058365 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1152/2314] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:04:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:04:30.062877 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00b5f72-5d24-4efc-852c-3eda51db5f0d admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:30.073874 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00b5f72-5d24-4efc-852c-3eda51db5f0d admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:30.078799 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48db216b-4a2d-4ed8-8ffe-8ec462e21c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:30.079646 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48db216b-4a2d-4ed8-8ffe-8ec462e21c81 admin admin] REQUEST_METHOD: `PUT` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:30.079929 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48db216b-4a2d-4ed8-8ffe-8ec462e21c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:30.080206 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48db216b-4a2d-4ed8-8ffe-8ec462e21c81 admin admin] PATH_INFO: `/v3/projects/29a0a0c03ad94249a93ad6f548b5a9ec/users/6530410c7b6c449db3870d7a500bbe05/roles/b674fd0a592e4e0cada811bba5ceea4d` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:30.105424 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48db216b-4a2d-4ed8-8ffe-8ec462e21c81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29a0a0c03ad94249a93ad6f548b5a9ec, user_id=6530410c7b6c449db3870d7a500bbe05, role_id=b674fd0a592e4e0cada811bba5ceea4d)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:30.108437 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48db216b-4a2d-4ed8-8ffe-8ec462e21c81 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:30.201080 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1163/2315] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 09:04:30 2022] PUT /identity/v3/projects/29a0a0c03ad94249a93ad6f548b5a9ec/users/6530410c7b6c449db3870d7a500bbe05/roles/b674fd0a592e4e0cada811bba5ceea4d => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:04:30.213801 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a7bb30-da60-4532-9b64-ae8154ed7b2d None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:30.214116 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a7bb30-da60-4532-9b64-ae8154ed7b2d None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:30.214407 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a7bb30-da60-4532-9b64-ae8154ed7b2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:30.247626 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-76a7bb30-da60-4532-9b64-ae8154ed7b2d None None] MFA Rules not processed for user `6530410c7b6c449db3870d7a500bbe05`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:04:30.248957 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-76a7bb30-da60-4532-9b64-ae8154ed7b2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:30.337984 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1153/2316] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:04:30 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:04:30.339427 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48db216b-4a2d-4ed8-8ffe-8ec462e21c81 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:30.418133 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:04:30.424014 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48db216b-4a2d-4ed8-8ffe-8ec462e21c81 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:30.429205 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136ead15-76a6-4ff1-9582-30c0e662514e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:30.429845 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136ead15-76a6-4ff1-9582-30c0e662514e admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:30.430129 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136ead15-76a6-4ff1-9582-30c0e662514e admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:30.430406 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136ead15-76a6-4ff1-9582-30c0e662514e admin admin] PATH_INFO: `/v3/projects` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:30.431104 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136ead15-76a6-4ff1-9582-30c0e662514e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:30.433149 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136ead15-76a6-4ff1-9582-30c0e662514e admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:30.448369 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e793fc-67a1-43ac-a8c3-9a9dcb383ef8 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:30.449096 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e793fc-67a1-43ac-a8c3-9a9dcb383ef8 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:30.449761 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e793fc-67a1-43ac-a8c3-9a9dcb383ef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:30.472612 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-17e793fc-67a1-43ac-a8c3-9a9dcb383ef8 None None] MFA Rules not processed for user `f5d7aee76519492e81b0048edebba36f`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:04:30.473860 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-17e793fc-67a1-43ac-a8c3-9a9dcb383ef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:30.534501 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1154/2317] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:04:30 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:04:30.557690 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1164/2318] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 09:04:30 2022] POST /identity/v3/projects => generated 355 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:04:30.565878 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e793fc-67a1-43ac-a8c3-9a9dcb383ef8 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:30.582534 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e793fc-67a1-43ac-a8c3-9a9dcb383ef8 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:30.639527 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:04:30.680414 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fc5e38-acdb-4ce4-805e-cb077224c5b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:30.681071 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fc5e38-acdb-4ce4-805e-cb077224c5b4 admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:30.681347 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fc5e38-acdb-4ce4-805e-cb077224c5b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:30.681615 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fc5e38-acdb-4ce4-805e-cb077224c5b4 admin admin] PATH_INFO: `/v3/users` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:30.682249 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fc5e38-acdb-4ce4-805e-cb077224c5b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:30.684206 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fc5e38-acdb-4ce4-805e-cb077224c5b4 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:30.780067 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1155/2319] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 09:04:30 2022] POST /identity/v3/users => generated 284 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:04:30.781144 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136ead15-76a6-4ff1-9582-30c0e662514e admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:30.792267 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136ead15-76a6-4ff1-9582-30c0e662514e admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:30.797213 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79744880-dd13-4646-aab7-c455924dc9fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:30.797964 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79744880-dd13-4646-aab7-c455924dc9fd admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:30.798310 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79744880-dd13-4646-aab7-c455924dc9fd admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:30.798651 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79744880-dd13-4646-aab7-c455924dc9fd admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:30.799374 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79744880-dd13-4646-aab7-c455924dc9fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:30.801061 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79744880-dd13-4646-aab7-c455924dc9fd admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:30.812829 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1165/2320] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:04:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:04:30.816739 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fc5e38-acdb-4ce4-805e-cb077224c5b4 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:30.832487 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fc5e38-acdb-4ce4-805e-cb077224c5b4 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:30.838945 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf93d547-7bc9-4f46-a1f5-f192bdf2efdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:30.839626 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf93d547-7bc9-4f46-a1f5-f192bdf2efdb admin admin] REQUEST_METHOD: `PUT` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:30.839898 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf93d547-7bc9-4f46-a1f5-f192bdf2efdb admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:30.840170 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf93d547-7bc9-4f46-a1f5-f192bdf2efdb admin admin] PATH_INFO: `/v3/projects/ff072aeeff664deeae03fc8e2e54cf83/users/0496281912b04c3dbad2e7c9df67ee3c/roles/b870cd1acaa744e1ae1d58b9ac283821` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:30.857588 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf93d547-7bc9-4f46-a1f5-f192bdf2efdb admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff072aeeff664deeae03fc8e2e54cf83, user_id=0496281912b04c3dbad2e7c9df67ee3c, role_id=b870cd1acaa744e1ae1d58b9ac283821)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:30.860278 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf93d547-7bc9-4f46-a1f5-f192bdf2efdb admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:30.953651 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1156/2321] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 09:04:30 2022] PUT /identity/v3/projects/ff072aeeff664deeae03fc8e2e54cf83/users/0496281912b04c3dbad2e7c9df67ee3c/roles/b870cd1acaa744e1ae1d58b9ac283821 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:04:30.957680 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79744880-dd13-4646-aab7-c455924dc9fd admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:30.973056 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79744880-dd13-4646-aab7-c455924dc9fd admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:31.076992 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6902af2c-af01-48df-860f-7fbf8d51850c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:31.079056 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6902af2c-af01-48df-860f-7fbf8d51850c admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:31.079056 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6902af2c-af01-48df-860f-7fbf8d51850c admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:31.079056 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6902af2c-af01-48df-860f-7fbf8d51850c admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:31.079056 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6902af2c-af01-48df-860f-7fbf8d51850c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:31.081098 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6902af2c-af01-48df-860f-7fbf8d51850c admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:31.102582 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1166/2322] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:04:30 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:04:31.106768 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf93d547-7bc9-4f46-a1f5-f192bdf2efdb admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:31.119130 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf93d547-7bc9-4f46-a1f5-f192bdf2efdb admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:31.125069 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ac0fd6-274c-406a-861a-be8088214891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:31.125754 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ac0fd6-274c-406a-861a-be8088214891 admin admin] REQUEST_METHOD: `PUT` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:31.126031 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ac0fd6-274c-406a-861a-be8088214891 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:31.126310 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ac0fd6-274c-406a-861a-be8088214891 admin admin] PATH_INFO: `/v3/projects/ff072aeeff664deeae03fc8e2e54cf83/users/0496281912b04c3dbad2e7c9df67ee3c/roles/b870cd1acaa744e1ae1d58b9ac283821` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:31.129059 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ac0fd6-274c-406a-861a-be8088214891 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff072aeeff664deeae03fc8e2e54cf83, user_id=0496281912b04c3dbad2e7c9df67ee3c, role_id=b870cd1acaa744e1ae1d58b9ac283821)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:31.184004 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ac0fd6-274c-406a-861a-be8088214891 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:31.184004 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1157/2323] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 09:04:31 2022] PUT /identity/v3/projects/ff072aeeff664deeae03fc8e2e54cf83/users/0496281912b04c3dbad2e7c9df67ee3c/roles/b870cd1acaa744e1ae1d58b9ac283821 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:04:31.198441 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed3b47c-bf70-404a-adca-c6725748a763 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:31.202685 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed3b47c-bf70-404a-adca-c6725748a763 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:31.202685 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed3b47c-bf70-404a-adca-c6725748a763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:31.232698 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-7ed3b47c-bf70-404a-adca-c6725748a763 None None] MFA Rules not processed for user `0496281912b04c3dbad2e7c9df67ee3c`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:04:31.233950 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-7ed3b47c-bf70-404a-adca-c6725748a763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:31.311262 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1167/2324] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:04:31 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:04:31.323247 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ac0fd6-274c-406a-861a-be8088214891 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:31.325282 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-57ac0fd6-274c-406a-861a-be8088214891 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:31.418326 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ac0fd6-274c-406a-861a-be8088214891 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:31.422156 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ddc6ba-d72d-4bb4-bf57-084c703685d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbb9ac667f04c35a63b2689df18efed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:31.422759 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ddc6ba-d72d-4bb4-bf57-084c703685d3 service cinder] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:31.423034 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ddc6ba-d72d-4bb4-bf57-084c703685d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:31.423302 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ddc6ba-d72d-4bb4-bf57-084c703685d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:31.442438 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ddc6ba-d72d-4bb4-bf57-084c703685d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:31.445680 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ddc6ba-d72d-4bb4-bf57-084c703685d3 service cinder] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:31.467252 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1158/2325] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:04:31 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:04:31.723503 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed3b47c-bf70-404a-adca-c6725748a763 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:31.735427 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed3b47c-bf70-404a-adca-c6725748a763 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:31.741334 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d836d0a6-b405-4ba4-9a18-0dcba319848c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbb9ac667f04c35a63b2689df18efed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:31.741334 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d836d0a6-b405-4ba4-9a18-0dcba319848c service cinder] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:31.741334 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d836d0a6-b405-4ba4-9a18-0dcba319848c service cinder] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:31.741334 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d836d0a6-b405-4ba4-9a18-0dcba319848c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:31.792081 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:04:31.803201 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d836d0a6-b405-4ba4-9a18-0dcba319848c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:31.804726 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d836d0a6-b405-4ba4-9a18-0dcba319848c service cinder] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:31.814980 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1168/2326] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:04:31 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:04:41.457893 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=63,keystone:INSERT=5 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:04:42.084459 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=5 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:04:56.632624 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ddc6ba-d72d-4bb4-bf57-084c703685d3 service cinder] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:56.701350 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ddc6ba-d72d-4bb4-bf57-084c703685d3 service cinder] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:56.706108 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6363d2-ae29-440c-892e-8b74292dfde1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:56.706755 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6363d2-ae29-440c-892e-8b74292dfde1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:56.707036 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6363d2-ae29-440c-892e-8b74292dfde1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:56.707316 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6363d2-ae29-440c-892e-8b74292dfde1 admin admin] PATH_INFO: `/v3/users/6530410c7b6c449db3870d7a500bbe05` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:56.708498 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6363d2-ae29-440c-892e-8b74292dfde1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6530410c7b6c449db3870d7a500bbe05)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:56.710528 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6363d2-ae29-440c-892e-8b74292dfde1 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:56.922775 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-cd6363d2-ae29-440c-892e-8b74292dfde1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6530410c7b6c449db3870d7a500bbe05'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:56.923419 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-cd6363d2-ae29-440c-892e-8b74292dfde1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6530410c7b6c449db3870d7a500bbe05'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:56.924027 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-cd6363d2-ae29-440c-892e-8b74292dfde1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6530410c7b6c449db3870d7a500bbe05'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:56.937349 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-cd6363d2-ae29-440c-892e-8b74292dfde1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6530410c7b6c449db3870d7a500bbe05'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:57.026780 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-cd6363d2-ae29-440c-892e-8b74292dfde1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6530410c7b6c449db3870d7a500bbe05'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:57.041962 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1159/2327] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 09:04:56 2022] DELETE /identity/v3/users/6530410c7b6c449db3870d7a500bbe05 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:04:57.046658 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b847c7-e7b5-4d79-a287-6fcc723d33eb None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:57.047186 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b847c7-e7b5-4d79-a287-6fcc723d33eb None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:57.047186 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b847c7-e7b5-4d79-a287-6fcc723d33eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:57.067138 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-75b847c7-e7b5-4d79-a287-6fcc723d33eb None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:04:57.068361 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-75b847c7-e7b5-4d79-a287-6fcc723d33eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:57.130831 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1169/2328] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:04:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:04:57.140807 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6363d2-ae29-440c-892e-8b74292dfde1 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:57.142251 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-cd6363d2-ae29-440c-892e-8b74292dfde1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:57.220013 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6363d2-ae29-440c-892e-8b74292dfde1 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:57.223698 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8c64a9-43c3-4cef-b1b5-4cdf79e88f85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:57.224579 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8c64a9-43c3-4cef-b1b5-4cdf79e88f85 service neutron] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:57.224724 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8c64a9-43c3-4cef-b1b5-4cdf79e88f85 service neutron] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:57.224988 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8c64a9-43c3-4cef-b1b5-4cdf79e88f85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:57.235816 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8c64a9-43c3-4cef-b1b5-4cdf79e88f85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:57.237278 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8c64a9-43c3-4cef-b1b5-4cdf79e88f85 service neutron] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:57.247302 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1160/2329] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:04:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:04:57.676240 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b847c7-e7b5-4d79-a287-6fcc723d33eb None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:57.678563 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-75b847c7-e7b5-4d79-a287-6fcc723d33eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:57.700308 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b847c7-e7b5-4d79-a287-6fcc723d33eb None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:57.706401 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7cbfba-749e-4669-91ae-5c2261dc47a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:57.706401 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7cbfba-749e-4669-91ae-5c2261dc47a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:57.707061 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7cbfba-749e-4669-91ae-5c2261dc47a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:57.707061 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7cbfba-749e-4669-91ae-5c2261dc47a6 admin admin] PATH_INFO: `/v3/projects/29a0a0c03ad94249a93ad6f548b5a9ec` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:57.708077 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7cbfba-749e-4669-91ae-5c2261dc47a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=29a0a0c03ad94249a93ad6f548b5a9ec)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:57.710706 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7cbfba-749e-4669-91ae-5c2261dc47a6 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:57.806092 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-ba7cbfba-749e-4669-91ae-5c2261dc47a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29a0a0c03ad94249a93ad6f548b5a9ec', 'request_id': 'req-ba7cbfba-749e-4669-91ae-5c2261dc47a6'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:57.812566 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-ba7cbfba-749e-4669-91ae-5c2261dc47a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29a0a0c03ad94249a93ad6f548b5a9ec', 'request_id': 'req-ba7cbfba-749e-4669-91ae-5c2261dc47a6'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:57.878746 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-ba7cbfba-749e-4669-91ae-5c2261dc47a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29a0a0c03ad94249a93ad6f548b5a9ec', 'request_id': 'req-ba7cbfba-749e-4669-91ae-5c2261dc47a6'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:57.893119 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-ba7cbfba-749e-4669-91ae-5c2261dc47a6 admin admin] The token cache is being invalidate because project 29a0a0c03ad94249a93ad6f548b5a9ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:04:57.893412 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-ba7cbfba-749e-4669-91ae-5c2261dc47a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:57.894145 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-ba7cbfba-749e-4669-91ae-5c2261dc47a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:57.895672 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1170/2330] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 09:04:57 2022] DELETE /identity/v3/projects/29a0a0c03ad94249a93ad6f548b5a9ec => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:04:57.900149 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8c64a9-43c3-4cef-b1b5-4cdf79e88f85 service neutron] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:57.911163 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8c64a9-43c3-4cef-b1b5-4cdf79e88f85 service neutron] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:57.925598 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:04:57.986769 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b7821a-ee8f-4c4a-9a4d-a92b767acbd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:57.987445 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b7821a-ee8f-4c4a-9a4d-a92b767acbd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:57.988016 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b7821a-ee8f-4c4a-9a4d-a92b767acbd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:57.988358 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b7821a-ee8f-4c4a-9a4d-a92b767acbd2 admin admin] PATH_INFO: `/v3/users/0496281912b04c3dbad2e7c9df67ee3c` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:57.990085 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b7821a-ee8f-4c4a-9a4d-a92b767acbd2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0496281912b04c3dbad2e7c9df67ee3c)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:57.992610 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b7821a-ee8f-4c4a-9a4d-a92b767acbd2 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:58.178700 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-e5b7821a-ee8f-4c4a-9a4d-a92b767acbd2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0496281912b04c3dbad2e7c9df67ee3c'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:58.179916 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-e5b7821a-ee8f-4c4a-9a4d-a92b767acbd2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0496281912b04c3dbad2e7c9df67ee3c'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:58.180518 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-e5b7821a-ee8f-4c4a-9a4d-a92b767acbd2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0496281912b04c3dbad2e7c9df67ee3c'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:58.193744 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-e5b7821a-ee8f-4c4a-9a4d-a92b767acbd2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0496281912b04c3dbad2e7c9df67ee3c'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:58.489743 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-e5b7821a-ee8f-4c4a-9a4d-a92b767acbd2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0496281912b04c3dbad2e7c9df67ee3c'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:58.505308 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1161/2331] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 09:04:57 2022] DELETE /identity/v3/users/0496281912b04c3dbad2e7c9df67ee3c => generated 0 bytes in 606 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:04:59.046464 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7cbfba-749e-4669-91ae-5c2261dc47a6 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:59.051393 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-ba7cbfba-749e-4669-91ae-5c2261dc47a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:59.069983 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7cbfba-749e-4669-91ae-5c2261dc47a6 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:59.142944 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae2358a-0716-49be-a66d-2cd4c46904b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:59.143596 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae2358a-0716-49be-a66d-2cd4c46904b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:59.143875 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae2358a-0716-49be-a66d-2cd4c46904b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:59.144146 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae2358a-0716-49be-a66d-2cd4c46904b0 admin admin] PATH_INFO: `/v3/projects/ff072aeeff664deeae03fc8e2e54cf83` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:59.145376 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae2358a-0716-49be-a66d-2cd4c46904b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff072aeeff664deeae03fc8e2e54cf83)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:59.147915 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae2358a-0716-49be-a66d-2cd4c46904b0 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:59.258259 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:04:59.277290 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-bae2358a-0716-49be-a66d-2cd4c46904b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff072aeeff664deeae03fc8e2e54cf83', 'request_id': 'req-bae2358a-0716-49be-a66d-2cd4c46904b0'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:59.283956 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-bae2358a-0716-49be-a66d-2cd4c46904b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff072aeeff664deeae03fc8e2e54cf83', 'request_id': 'req-bae2358a-0716-49be-a66d-2cd4c46904b0'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:59.381209 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-bae2358a-0716-49be-a66d-2cd4c46904b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff072aeeff664deeae03fc8e2e54cf83', 'request_id': 'req-bae2358a-0716-49be-a66d-2cd4c46904b0'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:59.399455 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-bae2358a-0716-49be-a66d-2cd4c46904b0 admin admin] The token cache is being invalidate because project ff072aeeff664deeae03fc8e2e54cf83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:04:59.399734 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-bae2358a-0716-49be-a66d-2cd4c46904b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:59.400442 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-bae2358a-0716-49be-a66d-2cd4c46904b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:59.401996 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1171/2332] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 09:04:59 2022] DELETE /identity/v3/projects/ff072aeeff664deeae03fc8e2e54cf83 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:04:59.407045 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83959b14-dbd5-48c5-bac4-75343be40f34 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:59.407977 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83959b14-dbd5-48c5-bac4-75343be40f34 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:59.407977 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83959b14-dbd5-48c5-bac4-75343be40f34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:59.430705 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-83959b14-dbd5-48c5-bac4-75343be40f34 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:04:59.432021 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-83959b14-dbd5-48c5-bac4-75343be40f34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:59.501319 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1162/2333] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:04:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:04:59.511701 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822a3e8d-c019-411d-b13a-fc5c04ad4bf2 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:59.512080 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822a3e8d-c019-411d-b13a-fc5c04ad4bf2 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:59.512405 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822a3e8d-c019-411d-b13a-fc5c04ad4bf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:59.534095 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-822a3e8d-c019-411d-b13a-fc5c04ad4bf2 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:04:59.535359 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-822a3e8d-c019-411d-b13a-fc5c04ad4bf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:59.544118 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1172/2334] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:04:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:04:59.548560 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83959b14-dbd5-48c5-bac4-75343be40f34 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:59.559553 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83959b14-dbd5-48c5-bac4-75343be40f34 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:59.564526 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4714fc92-d20f-4d54-a3b8-29cb06ece728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:59.565178 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4714fc92-d20f-4d54-a3b8-29cb06ece728 admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:59.565467 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4714fc92-d20f-4d54-a3b8-29cb06ece728 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:59.565740 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4714fc92-d20f-4d54-a3b8-29cb06ece728 admin admin] PATH_INFO: `/v3/domains` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:59.566399 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4714fc92-d20f-4d54-a3b8-29cb06ece728 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:04:59.566709 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4714fc92-d20f-4d54-a3b8-29cb06ece728 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:59.568359 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4714fc92-d20f-4d54-a3b8-29cb06ece728 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:59.581893 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1163/2335] 10.222.0.55 () {58 vars in 1153 bytes} [Wed Aug 24 09:04:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:04:59.586558 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822a3e8d-c019-411d-b13a-fc5c04ad4bf2 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:59.598688 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822a3e8d-c019-411d-b13a-fc5c04ad4bf2 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:59.607336 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03f5172-5106-474d-ba2c-db4b0e16c1b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:59.608329 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03f5172-5106-474d-ba2c-db4b0e16c1b6 admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:59.609014 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03f5172-5106-474d-ba2c-db4b0e16c1b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:59.609298 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03f5172-5106-474d-ba2c-db4b0e16c1b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:59.609955 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03f5172-5106-474d-ba2c-db4b0e16c1b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:59.611899 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03f5172-5106-474d-ba2c-db4b0e16c1b6 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:59.703311 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1173/2336] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 09:04:59 2022] POST /identity/v3/projects => generated 363 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:04:59.709359 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4714fc92-d20f-4d54-a3b8-29cb06ece728 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:59.720369 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4714fc92-d20f-4d54-a3b8-29cb06ece728 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:59.734965 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:04:59.790508 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78862063-88e8-4432-b2de-c58b5519ee23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:59.791645 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78862063-88e8-4432-b2de-c58b5519ee23 admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:59.792119 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78862063-88e8-4432-b2de-c58b5519ee23 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:59.792692 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78862063-88e8-4432-b2de-c58b5519ee23 admin admin] PATH_INFO: `/v3/users` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:59.793767 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78862063-88e8-4432-b2de-c58b5519ee23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:59.797382 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78862063-88e8-4432-b2de-c58b5519ee23 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:59.907828 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1164/2337] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 09:04:59 2022] POST /identity/v3/users => generated 282 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:04:59.913500 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03f5172-5106-474d-ba2c-db4b0e16c1b6 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:59.932073 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03f5172-5106-474d-ba2c-db4b0e16c1b6 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:59.939133 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d9c4cc-95b2-4665-945d-99ecf4984c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:59.940323 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d9c4cc-95b2-4665-945d-99ecf4984c6b admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:59.940883 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d9c4cc-95b2-4665-945d-99ecf4984c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:59.941386 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d9c4cc-95b2-4665-945d-99ecf4984c6b admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:59.942629 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d9c4cc-95b2-4665-945d-99ecf4984c6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:59.945873 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d9c4cc-95b2-4665-945d-99ecf4984c6b admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:59.963273 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1174/2338] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:04:59 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:04:59.967718 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78862063-88e8-4432-b2de-c58b5519ee23 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:59.979653 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78862063-88e8-4432-b2de-c58b5519ee23 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:59.984697 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4862dc-1f02-4dca-8b05-d22746e13d55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:59.985353 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4862dc-1f02-4dca-8b05-d22746e13d55 admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:59.985637 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4862dc-1f02-4dca-8b05-d22746e13d55 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:59.985913 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4862dc-1f02-4dca-8b05-d22746e13d55 admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:59.986576 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4862dc-1f02-4dca-8b05-d22746e13d55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:59.988223 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4862dc-1f02-4dca-8b05-d22746e13d55 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:59.999462 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1165/2339] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:04:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:05:00.005849 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d9c4cc-95b2-4665-945d-99ecf4984c6b admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:00.016857 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d9c4cc-95b2-4665-945d-99ecf4984c6b admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:00.022142 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d0701b-416f-43b5-b070-28f3d2cc5189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:00.023299 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d0701b-416f-43b5-b070-28f3d2cc5189 admin admin] REQUEST_METHOD: `PUT` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:00.023580 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d0701b-416f-43b5-b070-28f3d2cc5189 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:00.023857 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d0701b-416f-43b5-b070-28f3d2cc5189 admin admin] PATH_INFO: `/v3/projects/c6214bcb90d045eab095f0db3bc4ef1e/users/e360552c44d04234a429918d0533f7df/roles/b674fd0a592e4e0cada811bba5ceea4d` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:00.041544 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d0701b-416f-43b5-b070-28f3d2cc5189 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6214bcb90d045eab095f0db3bc4ef1e, user_id=e360552c44d04234a429918d0533f7df, role_id=b674fd0a592e4e0cada811bba5ceea4d)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:00.044260 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d0701b-416f-43b5-b070-28f3d2cc5189 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:00.153138 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1175/2340] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 09:05:00 2022] PUT /identity/v3/projects/c6214bcb90d045eab095f0db3bc4ef1e/users/e360552c44d04234a429918d0533f7df/roles/b674fd0a592e4e0cada811bba5ceea4d => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:05:00.166203 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdeb518-c6fe-4823-8300-815a7ac7d38a None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:00.166553 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdeb518-c6fe-4823-8300-815a7ac7d38a None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:00.166865 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdeb518-c6fe-4823-8300-815a7ac7d38a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:00.205966 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-ebdeb518-c6fe-4823-8300-815a7ac7d38a None None] MFA Rules not processed for user `e360552c44d04234a429918d0533f7df`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:05:00.207945 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-ebdeb518-c6fe-4823-8300-815a7ac7d38a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:00.308911 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1166/2341] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:05:00 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:05:00.315602 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d0701b-416f-43b5-b070-28f3d2cc5189 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:00.394385 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d0701b-416f-43b5-b070-28f3d2cc5189 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:00.400363 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982f7938-4a7a-43e3-8295-377e49ddf43e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:00.401090 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982f7938-4a7a-43e3-8295-377e49ddf43e admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:00.401450 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982f7938-4a7a-43e3-8295-377e49ddf43e admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:00.401770 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982f7938-4a7a-43e3-8295-377e49ddf43e admin admin] PATH_INFO: `/v3/projects` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:00.402631 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982f7938-4a7a-43e3-8295-377e49ddf43e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:00.404969 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982f7938-4a7a-43e3-8295-377e49ddf43e admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:00.498455 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1176/2342] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 09:05:00 2022] POST /identity/v3/projects => generated 363 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:05:00.504152 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdeb518-c6fe-4823-8300-815a7ac7d38a None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:00.517955 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdeb518-c6fe-4823-8300-815a7ac7d38a None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:00.588820 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7d3ce8-4bcc-4f6a-b45f-8fcfbebcfba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:00.589457 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7d3ce8-4bcc-4f6a-b45f-8fcfbebcfba7 admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:00.589733 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7d3ce8-4bcc-4f6a-b45f-8fcfbebcfba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:00.590017 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7d3ce8-4bcc-4f6a-b45f-8fcfbebcfba7 admin admin] PATH_INFO: `/v3/users` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:00.590721 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7d3ce8-4bcc-4f6a-b45f-8fcfbebcfba7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:00.592736 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7d3ce8-4bcc-4f6a-b45f-8fcfbebcfba7 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:00.718839 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1167/2343] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 09:05:00 2022] POST /identity/v3/users => generated 288 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:05:00.724258 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982f7938-4a7a-43e3-8295-377e49ddf43e admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:00.739245 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982f7938-4a7a-43e3-8295-377e49ddf43e admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:00.744482 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bf187f-6f19-43b6-8ac8-beb8bdf8ed96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:00.745303 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf187f-6f19-43b6-8ac8-beb8bdf8ed96 admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:00.745632 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf187f-6f19-43b6-8ac8-beb8bdf8ed96 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:00.745952 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf187f-6f19-43b6-8ac8-beb8bdf8ed96 admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:00.746691 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bf187f-6f19-43b6-8ac8-beb8bdf8ed96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:00.748653 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bf187f-6f19-43b6-8ac8-beb8bdf8ed96 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:00.762130 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1177/2344] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:05:00 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:05:00.767726 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7d3ce8-4bcc-4f6a-b45f-8fcfbebcfba7 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:00.779743 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7d3ce8-4bcc-4f6a-b45f-8fcfbebcfba7 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:00.785649 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdb4f5e-0ff7-4700-9abf-a6af113bb616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:00.786490 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdb4f5e-0ff7-4700-9abf-a6af113bb616 admin admin] REQUEST_METHOD: `PUT` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:00.786781 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdb4f5e-0ff7-4700-9abf-a6af113bb616 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:00.787062 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdb4f5e-0ff7-4700-9abf-a6af113bb616 admin admin] PATH_INFO: `/v3/projects/d3cddf601b50457f81a9341b6abaca9e/users/0244e8840057417fa7f2ca909e2da57f/roles/b870cd1acaa744e1ae1d58b9ac283821` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:00.803018 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:05:00.806670 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdb4f5e-0ff7-4700-9abf-a6af113bb616 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3cddf601b50457f81a9341b6abaca9e, user_id=0244e8840057417fa7f2ca909e2da57f, role_id=b870cd1acaa744e1ae1d58b9ac283821)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:00.809805 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdb4f5e-0ff7-4700-9abf-a6af113bb616 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:00.942267 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1168/2345] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 09:05:00 2022] PUT /identity/v3/projects/d3cddf601b50457f81a9341b6abaca9e/users/0244e8840057417fa7f2ca909e2da57f/roles/b870cd1acaa744e1ae1d58b9ac283821 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:05:00.946987 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bf187f-6f19-43b6-8ac8-beb8bdf8ed96 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:00.958047 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bf187f-6f19-43b6-8ac8-beb8bdf8ed96 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:00.985575 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:05:01.030675 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778d3cef-a2aa-40f6-98b9-0247ecc9c4bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:01.031199 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778d3cef-a2aa-40f6-98b9-0247ecc9c4bb admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:01.031484 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778d3cef-a2aa-40f6-98b9-0247ecc9c4bb admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:01.031761 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778d3cef-a2aa-40f6-98b9-0247ecc9c4bb admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:01.032713 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778d3cef-a2aa-40f6-98b9-0247ecc9c4bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:01.034355 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778d3cef-a2aa-40f6-98b9-0247ecc9c4bb admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:01.046366 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1178/2346] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:05:00 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:05:01.051256 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdb4f5e-0ff7-4700-9abf-a6af113bb616 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:01.066539 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdb4f5e-0ff7-4700-9abf-a6af113bb616 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:01.074197 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339fed23-93c6-428c-b9d2-775a2dd7ab8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:01.074197 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339fed23-93c6-428c-b9d2-775a2dd7ab8a admin admin] REQUEST_METHOD: `PUT` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:01.074197 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339fed23-93c6-428c-b9d2-775a2dd7ab8a admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:01.074197 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339fed23-93c6-428c-b9d2-775a2dd7ab8a admin admin] PATH_INFO: `/v3/projects/d3cddf601b50457f81a9341b6abaca9e/users/0244e8840057417fa7f2ca909e2da57f/roles/b870cd1acaa744e1ae1d58b9ac283821` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:01.075555 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339fed23-93c6-428c-b9d2-775a2dd7ab8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3cddf601b50457f81a9341b6abaca9e, user_id=0244e8840057417fa7f2ca909e2da57f, role_id=b870cd1acaa744e1ae1d58b9ac283821)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:01.078760 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339fed23-93c6-428c-b9d2-775a2dd7ab8a admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:01.093305 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1169/2347] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 09:05:01 2022] PUT /identity/v3/projects/d3cddf601b50457f81a9341b6abaca9e/users/0244e8840057417fa7f2ca909e2da57f/roles/b870cd1acaa744e1ae1d58b9ac283821 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:05:01.104666 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3341b1a7-0133-48d1-b8be-bf64a6ffd68e None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:01.104943 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3341b1a7-0133-48d1-b8be-bf64a6ffd68e None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:01.105218 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3341b1a7-0133-48d1-b8be-bf64a6ffd68e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:01.136236 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-3341b1a7-0133-48d1-b8be-bf64a6ffd68e None None] MFA Rules not processed for user `0244e8840057417fa7f2ca909e2da57f`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:05:01.137626 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-3341b1a7-0133-48d1-b8be-bf64a6ffd68e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:01.214827 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1179/2348] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:05:01 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:05:01.227805 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339fed23-93c6-428c-b9d2-775a2dd7ab8a admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:01.229663 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-339fed23-93c6-428c-b9d2-775a2dd7ab8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:01.293327 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339fed23-93c6-428c-b9d2-775a2dd7ab8a admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:01.297730 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230a7d26-2de6-4795-8fd4-b760eabb0265 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbb9ac667f04c35a63b2689df18efed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:01.298355 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230a7d26-2de6-4795-8fd4-b760eabb0265 service cinder] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:01.298630 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230a7d26-2de6-4795-8fd4-b760eabb0265 service cinder] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:01.298896 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230a7d26-2de6-4795-8fd4-b760eabb0265 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:01.309009 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230a7d26-2de6-4795-8fd4-b760eabb0265 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:01.310502 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230a7d26-2de6-4795-8fd4-b760eabb0265 service cinder] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:01.321851 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1170/2349] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:05:01 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:05:01.592700 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3341b1a7-0133-48d1-b8be-bf64a6ffd68e None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:01.608623 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3341b1a7-0133-48d1-b8be-bf64a6ffd68e None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:01.614619 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457a5eaa-3dbc-45b7-8d6c-c90f06cfb4b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbb9ac667f04c35a63b2689df18efed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:01.615385 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457a5eaa-3dbc-45b7-8d6c-c90f06cfb4b2 service cinder] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:01.615748 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457a5eaa-3dbc-45b7-8d6c-c90f06cfb4b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:01.616667 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457a5eaa-3dbc-45b7-8d6c-c90f06cfb4b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:01.683515 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457a5eaa-3dbc-45b7-8d6c-c90f06cfb4b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:01.685010 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457a5eaa-3dbc-45b7-8d6c-c90f06cfb4b2 service cinder] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:01.706210 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1180/2350] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:05:01 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:05:05.830571 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230a7d26-2de6-4795-8fd4-b760eabb0265 service cinder] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:05.898569 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230a7d26-2de6-4795-8fd4-b760eabb0265 service cinder] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:05.903371 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2135e4b6-f0eb-4aa1-93f2-1437b0847495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:05.904048 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2135e4b6-f0eb-4aa1-93f2-1437b0847495 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:05.904289 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2135e4b6-f0eb-4aa1-93f2-1437b0847495 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:05.904589 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2135e4b6-f0eb-4aa1-93f2-1437b0847495 admin admin] PATH_INFO: `/v3/users/e360552c44d04234a429918d0533f7df` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:05.905803 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2135e4b6-f0eb-4aa1-93f2-1437b0847495 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e360552c44d04234a429918d0533f7df)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:05.907984 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2135e4b6-f0eb-4aa1-93f2-1437b0847495 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:06.133473 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-2135e4b6-f0eb-4aa1-93f2-1437b0847495 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e360552c44d04234a429918d0533f7df'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:06.134192 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-2135e4b6-f0eb-4aa1-93f2-1437b0847495 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e360552c44d04234a429918d0533f7df'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:06.134794 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-2135e4b6-f0eb-4aa1-93f2-1437b0847495 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e360552c44d04234a429918d0533f7df'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:06.146746 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-2135e4b6-f0eb-4aa1-93f2-1437b0847495 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e360552c44d04234a429918d0533f7df'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:06.232272 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-2135e4b6-f0eb-4aa1-93f2-1437b0847495 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e360552c44d04234a429918d0533f7df'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:06.242247 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:05:06.246765 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1171/2351] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 09:05:05 2022] DELETE /identity/v3/users/e360552c44d04234a429918d0533f7df => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:05:06.251239 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca58baf-9bff-460e-bf9f-a1adc4106c32 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:06.251719 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca58baf-9bff-460e-bf9f-a1adc4106c32 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:06.251824 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca58baf-9bff-460e-bf9f-a1adc4106c32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:06.271519 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-eca58baf-9bff-460e-bf9f-a1adc4106c32 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:05:06.272762 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-eca58baf-9bff-460e-bf9f-a1adc4106c32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:06.314770 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:05:06.334354 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1181/2352] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:05:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:05:06.344330 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2135e4b6-f0eb-4aa1-93f2-1437b0847495 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:06.345796 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-2135e4b6-f0eb-4aa1-93f2-1437b0847495 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:06.414730 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2135e4b6-f0eb-4aa1-93f2-1437b0847495 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:06.418486 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e39dfc-c71e-4da1-8f3b-640f0305408d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:06.419126 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e39dfc-c71e-4da1-8f3b-640f0305408d service neutron] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:06.419412 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e39dfc-c71e-4da1-8f3b-640f0305408d service neutron] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:06.419665 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e39dfc-c71e-4da1-8f3b-640f0305408d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:06.430843 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e39dfc-c71e-4da1-8f3b-640f0305408d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:06.433223 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e39dfc-c71e-4da1-8f3b-640f0305408d service neutron] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:06.444067 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1172/2353] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:05:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:05:06.891423 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca58baf-9bff-460e-bf9f-a1adc4106c32 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:06.892909 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-eca58baf-9bff-460e-bf9f-a1adc4106c32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:06.907135 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca58baf-9bff-460e-bf9f-a1adc4106c32 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:06.912277 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b704eb5e-21a6-41f3-a04c-2ac2dd87b06e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:06.913546 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b704eb5e-21a6-41f3-a04c-2ac2dd87b06e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:06.913830 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b704eb5e-21a6-41f3-a04c-2ac2dd87b06e admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:06.914100 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b704eb5e-21a6-41f3-a04c-2ac2dd87b06e admin admin] PATH_INFO: `/v3/projects/c6214bcb90d045eab095f0db3bc4ef1e` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:06.915299 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b704eb5e-21a6-41f3-a04c-2ac2dd87b06e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6214bcb90d045eab095f0db3bc4ef1e)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:06.917532 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b704eb5e-21a6-41f3-a04c-2ac2dd87b06e admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:07.052590 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-b704eb5e-21a6-41f3-a04c-2ac2dd87b06e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6214bcb90d045eab095f0db3bc4ef1e', 'request_id': 'req-b704eb5e-21a6-41f3-a04c-2ac2dd87b06e'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:07.058457 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-b704eb5e-21a6-41f3-a04c-2ac2dd87b06e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6214bcb90d045eab095f0db3bc4ef1e', 'request_id': 'req-b704eb5e-21a6-41f3-a04c-2ac2dd87b06e'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:07.141212 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-b704eb5e-21a6-41f3-a04c-2ac2dd87b06e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6214bcb90d045eab095f0db3bc4ef1e', 'request_id': 'req-b704eb5e-21a6-41f3-a04c-2ac2dd87b06e'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:07.158875 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-b704eb5e-21a6-41f3-a04c-2ac2dd87b06e admin admin] The token cache is being invalidate because project c6214bcb90d045eab095f0db3bc4ef1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:05:07.159172 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-b704eb5e-21a6-41f3-a04c-2ac2dd87b06e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:07.159864 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-b704eb5e-21a6-41f3-a04c-2ac2dd87b06e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:07.161425 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1182/2354] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 09:05:06 2022] DELETE /identity/v3/projects/c6214bcb90d045eab095f0db3bc4ef1e => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:05:07.165808 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e39dfc-c71e-4da1-8f3b-640f0305408d service neutron] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:07.176647 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e39dfc-c71e-4da1-8f3b-640f0305408d service neutron] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:07.245744 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddb0746-fb6b-49b1-93ee-8ed670efc5cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:07.246374 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddb0746-fb6b-49b1-93ee-8ed670efc5cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:07.246654 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddb0746-fb6b-49b1-93ee-8ed670efc5cb admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:07.246927 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddb0746-fb6b-49b1-93ee-8ed670efc5cb admin admin] PATH_INFO: `/v3/users/0244e8840057417fa7f2ca909e2da57f` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:07.248083 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddb0746-fb6b-49b1-93ee-8ed670efc5cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=0244e8840057417fa7f2ca909e2da57f)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:07.250064 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddb0746-fb6b-49b1-93ee-8ed670efc5cb admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:07.504806 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-dddb0746-fb6b-49b1-93ee-8ed670efc5cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0244e8840057417fa7f2ca909e2da57f'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:07.505400 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-dddb0746-fb6b-49b1-93ee-8ed670efc5cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0244e8840057417fa7f2ca909e2da57f'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:07.506005 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-dddb0746-fb6b-49b1-93ee-8ed670efc5cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0244e8840057417fa7f2ca909e2da57f'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:07.518885 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-dddb0746-fb6b-49b1-93ee-8ed670efc5cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0244e8840057417fa7f2ca909e2da57f'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:07.604105 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-dddb0746-fb6b-49b1-93ee-8ed670efc5cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0244e8840057417fa7f2ca909e2da57f'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:07.618857 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1173/2355] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 09:05:07 2022] DELETE /identity/v3/users/0244e8840057417fa7f2ca909e2da57f => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:05:08.319549 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b704eb5e-21a6-41f3-a04c-2ac2dd87b06e admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:08.321137 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-b704eb5e-21a6-41f3-a04c-2ac2dd87b06e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:08.334962 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b704eb5e-21a6-41f3-a04c-2ac2dd87b06e admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:08.395912 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c8bf05-5f13-4a50-a231-a933d14bb664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:08.396576 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c8bf05-5f13-4a50-a231-a933d14bb664 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:08.396853 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c8bf05-5f13-4a50-a231-a933d14bb664 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:08.397619 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c8bf05-5f13-4a50-a231-a933d14bb664 admin admin] PATH_INFO: `/v3/projects/d3cddf601b50457f81a9341b6abaca9e` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:08.398787 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c8bf05-5f13-4a50-a231-a933d14bb664 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3cddf601b50457f81a9341b6abaca9e)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:08.400754 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c8bf05-5f13-4a50-a231-a933d14bb664 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:08.662419 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-d5c8bf05-5f13-4a50-a231-a933d14bb664 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3cddf601b50457f81a9341b6abaca9e', 'request_id': 'req-d5c8bf05-5f13-4a50-a231-a933d14bb664'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:08.667889 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-d5c8bf05-5f13-4a50-a231-a933d14bb664 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3cddf601b50457f81a9341b6abaca9e', 'request_id': 'req-d5c8bf05-5f13-4a50-a231-a933d14bb664'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:08.781073 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-d5c8bf05-5f13-4a50-a231-a933d14bb664 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3cddf601b50457f81a9341b6abaca9e', 'request_id': 'req-d5c8bf05-5f13-4a50-a231-a933d14bb664'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:08.789053 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:05:08.793566 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-d5c8bf05-5f13-4a50-a231-a933d14bb664 admin admin] The token cache is being invalidate because project d3cddf601b50457f81a9341b6abaca9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:05:08.793855 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-d5c8bf05-5f13-4a50-a231-a933d14bb664 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:08.794510 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-d5c8bf05-5f13-4a50-a231-a933d14bb664 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:08.795988 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1183/2356] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 09:05:08 2022] DELETE /identity/v3/projects/d3cddf601b50457f81a9341b6abaca9e => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:05:08.800908 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e8f656-1389-4bab-a864-68f4c8fc9a1b None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:08.801391 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e8f656-1389-4bab-a864-68f4c8fc9a1b None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:08.801496 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e8f656-1389-4bab-a864-68f4c8fc9a1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:08.822847 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-67e8f656-1389-4bab-a864-68f4c8fc9a1b None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:05:08.824065 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-67e8f656-1389-4bab-a864-68f4c8fc9a1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:08.841855 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:05:08.884526 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1174/2357] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:05:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:05:08.893181 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8cc8aa-8ebd-46ad-be81-4d364abcec77 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:08.893730 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8cc8aa-8ebd-46ad-be81-4d364abcec77 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:08.894325 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8cc8aa-8ebd-46ad-be81-4d364abcec77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:08.916214 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-1a8cc8aa-8ebd-46ad-be81-4d364abcec77 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:05:08.917674 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-1a8cc8aa-8ebd-46ad-be81-4d364abcec77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:08.925917 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1184/2358] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:05:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:05:08.930334 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e8f656-1389-4bab-a864-68f4c8fc9a1b None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:08.940666 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e8f656-1389-4bab-a864-68f4c8fc9a1b None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:08.946574 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bf09e7-ed44-4898-b41c-e3a3436d249d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:08.947195 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bf09e7-ed44-4898-b41c-e3a3436d249d admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:08.947501 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bf09e7-ed44-4898-b41c-e3a3436d249d admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:08.947740 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bf09e7-ed44-4898-b41c-e3a3436d249d admin admin] PATH_INFO: `/v3/domains` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:08.948427 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bf09e7-ed44-4898-b41c-e3a3436d249d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:05:08.948747 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bf09e7-ed44-4898-b41c-e3a3436d249d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:08.950402 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bf09e7-ed44-4898-b41c-e3a3436d249d admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:08.963842 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1175/2359] 10.222.0.55 () {58 vars in 1153 bytes} [Wed Aug 24 09:05:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:05:08.968345 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8cc8aa-8ebd-46ad-be81-4d364abcec77 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:08.977970 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8cc8aa-8ebd-46ad-be81-4d364abcec77 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:08.983360 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6937af-da4f-4bfc-bbb3-b7cf020536a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:08.983977 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6937af-da4f-4bfc-bbb3-b7cf020536a1 admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:08.984262 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6937af-da4f-4bfc-bbb3-b7cf020536a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:08.984568 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6937af-da4f-4bfc-bbb3-b7cf020536a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:08.985209 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6937af-da4f-4bfc-bbb3-b7cf020536a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:08.987132 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6937af-da4f-4bfc-bbb3-b7cf020536a1 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:09.103601 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1185/2360] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 09:05:08 2022] POST /identity/v3/projects => generated 391 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:05:09.109141 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bf09e7-ed44-4898-b41c-e3a3436d249d admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:09.120269 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bf09e7-ed44-4898-b41c-e3a3436d249d admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:09.179250 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e09a54-c089-4581-88c6-1faef6c8a964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:09.179859 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e09a54-c089-4581-88c6-1faef6c8a964 admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:09.180144 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e09a54-c089-4581-88c6-1faef6c8a964 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:09.180520 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e09a54-c089-4581-88c6-1faef6c8a964 admin admin] PATH_INFO: `/v3/users` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:09.181182 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e09a54-c089-4581-88c6-1faef6c8a964 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:09.183128 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e09a54-c089-4581-88c6-1faef6c8a964 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:09.306850 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1176/2361] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 09:05:09 2022] POST /identity/v3/users => generated 296 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:05:09.311092 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6937af-da4f-4bfc-bbb3-b7cf020536a1 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:09.322448 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6937af-da4f-4bfc-bbb3-b7cf020536a1 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:09.327541 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dceefd-27da-4aa5-8afa-288014483c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:09.328187 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dceefd-27da-4aa5-8afa-288014483c8f admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:09.328696 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dceefd-27da-4aa5-8afa-288014483c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:09.329195 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dceefd-27da-4aa5-8afa-288014483c8f admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:09.329731 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dceefd-27da-4aa5-8afa-288014483c8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:09.331295 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dceefd-27da-4aa5-8afa-288014483c8f admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:09.342165 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1186/2362] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:05:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:05:09.346547 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e09a54-c089-4581-88c6-1faef6c8a964 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:09.357732 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e09a54-c089-4581-88c6-1faef6c8a964 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:09.364096 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45df34cf-4e53-497d-96d2-ad8980ee48cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:09.364791 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45df34cf-4e53-497d-96d2-ad8980ee48cc admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:09.365140 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45df34cf-4e53-497d-96d2-ad8980ee48cc admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:09.365413 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45df34cf-4e53-497d-96d2-ad8980ee48cc admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:09.366743 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45df34cf-4e53-497d-96d2-ad8980ee48cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:09.368341 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45df34cf-4e53-497d-96d2-ad8980ee48cc admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:09.379858 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1177/2363] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:05:09 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 09:05:09.384419 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dceefd-27da-4aa5-8afa-288014483c8f admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:09.403559 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dceefd-27da-4aa5-8afa-288014483c8f admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:09.409937 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99b71b0-98da-46e3-9caf-0763b937cd44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:09.410629 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99b71b0-98da-46e3-9caf-0763b937cd44 admin admin] REQUEST_METHOD: `PUT` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:09.410906 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99b71b0-98da-46e3-9caf-0763b937cd44 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:09.411177 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99b71b0-98da-46e3-9caf-0763b937cd44 admin admin] PATH_INFO: `/v3/projects/0d0f7856ce1e49b8b70076841df4e302/users/bebf4d6902c14163844799c72563b172/roles/b674fd0a592e4e0cada811bba5ceea4d` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:09.427122 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99b71b0-98da-46e3-9caf-0763b937cd44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d0f7856ce1e49b8b70076841df4e302, user_id=bebf4d6902c14163844799c72563b172, role_id=b674fd0a592e4e0cada811bba5ceea4d)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:09.430451 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99b71b0-98da-46e3-9caf-0763b937cd44 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:09.513112 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1187/2364] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 09:05:09 2022] PUT /identity/v3/projects/0d0f7856ce1e49b8b70076841df4e302/users/bebf4d6902c14163844799c72563b172/roles/b674fd0a592e4e0cada811bba5ceea4d => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:05:09.523498 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06bcc7e-1a2f-4a8c-bca3-558f80d22801 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:09.523784 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06bcc7e-1a2f-4a8c-bca3-558f80d22801 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:09.524058 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06bcc7e-1a2f-4a8c-bca3-558f80d22801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:09.552929 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-d06bcc7e-1a2f-4a8c-bca3-558f80d22801 None None] MFA Rules not processed for user `bebf4d6902c14163844799c72563b172`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:05:09.554268 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-d06bcc7e-1a2f-4a8c-bca3-558f80d22801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:09.619908 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1178/2365] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:05:09 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:05:09.624372 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99b71b0-98da-46e3-9caf-0763b937cd44 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:09.691120 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99b71b0-98da-46e3-9caf-0763b937cd44 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:09.695926 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24024833-2019-40ae-9110-f20d20bed30a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:09.696561 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24024833-2019-40ae-9110-f20d20bed30a admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:09.696848 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24024833-2019-40ae-9110-f20d20bed30a admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:09.697118 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24024833-2019-40ae-9110-f20d20bed30a admin admin] PATH_INFO: `/v3/projects` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:09.697752 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24024833-2019-40ae-9110-f20d20bed30a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:09.699659 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24024833-2019-40ae-9110-f20d20bed30a admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:09.795038 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1188/2366] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 09:05:09 2022] POST /identity/v3/projects => generated 393 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:05:09.799632 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06bcc7e-1a2f-4a8c-bca3-558f80d22801 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:09.810971 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06bcc7e-1a2f-4a8c-bca3-558f80d22801 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:09.827469 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:05:09.875993 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdecaeb-b935-4571-93b8-af8cdb371310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:09.876639 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdecaeb-b935-4571-93b8-af8cdb371310 admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:09.876924 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdecaeb-b935-4571-93b8-af8cdb371310 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:09.877202 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdecaeb-b935-4571-93b8-af8cdb371310 admin admin] PATH_INFO: `/v3/users` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:09.877838 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdecaeb-b935-4571-93b8-af8cdb371310 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:09.879787 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdecaeb-b935-4571-93b8-af8cdb371310 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:10.015416 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1179/2367] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 09:05:09 2022] POST /identity/v3/users => generated 303 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:05:10.019584 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24024833-2019-40ae-9110-f20d20bed30a admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:10.031885 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24024833-2019-40ae-9110-f20d20bed30a admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:10.037041 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e2d5be-2a97-445e-9e0d-8f1a091eabca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:10.038227 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e2d5be-2a97-445e-9e0d-8f1a091eabca admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:10.038507 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e2d5be-2a97-445e-9e0d-8f1a091eabca admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:10.038776 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e2d5be-2a97-445e-9e0d-8f1a091eabca admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:10.039418 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e2d5be-2a97-445e-9e0d-8f1a091eabca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:10.041042 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e2d5be-2a97-445e-9e0d-8f1a091eabca admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:10.052397 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1189/2368] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:05:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:05:10.056442 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdecaeb-b935-4571-93b8-af8cdb371310 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:10.071032 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdecaeb-b935-4571-93b8-af8cdb371310 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:10.077527 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a79dc3-28f5-494f-8e1c-0ada43fdc72f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:10.078756 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a79dc3-28f5-494f-8e1c-0ada43fdc72f admin admin] REQUEST_METHOD: `PUT` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:10.079137 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a79dc3-28f5-494f-8e1c-0ada43fdc72f admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:10.079644 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a79dc3-28f5-494f-8e1c-0ada43fdc72f admin admin] PATH_INFO: `/v3/projects/7edc4b9e28c646a1a746e47ccde76985/users/0756689106704e11ad17144e1a367959/roles/b870cd1acaa744e1ae1d58b9ac283821` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:10.105029 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a79dc3-28f5-494f-8e1c-0ada43fdc72f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7edc4b9e28c646a1a746e47ccde76985, user_id=0756689106704e11ad17144e1a367959, role_id=b870cd1acaa744e1ae1d58b9ac283821)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:10.107753 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a79dc3-28f5-494f-8e1c-0ada43fdc72f admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:10.175746 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1180/2369] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 09:05:10 2022] PUT /identity/v3/projects/7edc4b9e28c646a1a746e47ccde76985/users/0756689106704e11ad17144e1a367959/roles/b870cd1acaa744e1ae1d58b9ac283821 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:05:10.180066 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e2d5be-2a97-445e-9e0d-8f1a091eabca admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:10.192140 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e2d5be-2a97-445e-9e0d-8f1a091eabca admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:10.260627 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6279c57-aa40-44f5-b1ac-32d01b74d010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:10.261247 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6279c57-aa40-44f5-b1ac-32d01b74d010 admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:10.261524 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6279c57-aa40-44f5-b1ac-32d01b74d010 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:10.261796 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6279c57-aa40-44f5-b1ac-32d01b74d010 admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:10.262440 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6279c57-aa40-44f5-b1ac-32d01b74d010 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:10.264070 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6279c57-aa40-44f5-b1ac-32d01b74d010 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:10.276084 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1190/2370] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:05:10 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:05:10.281054 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a79dc3-28f5-494f-8e1c-0ada43fdc72f admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:10.292658 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a79dc3-28f5-494f-8e1c-0ada43fdc72f admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:10.298007 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a50080-a6ae-41ae-8dbc-f582608caf67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:10.298687 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a50080-a6ae-41ae-8dbc-f582608caf67 admin admin] REQUEST_METHOD: `PUT` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:10.298967 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a50080-a6ae-41ae-8dbc-f582608caf67 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:10.299240 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a50080-a6ae-41ae-8dbc-f582608caf67 admin admin] PATH_INFO: `/v3/projects/7edc4b9e28c646a1a746e47ccde76985/users/0756689106704e11ad17144e1a367959/roles/b870cd1acaa744e1ae1d58b9ac283821` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:10.301237 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a50080-a6ae-41ae-8dbc-f582608caf67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7edc4b9e28c646a1a746e47ccde76985, user_id=0756689106704e11ad17144e1a367959, role_id=b870cd1acaa744e1ae1d58b9ac283821)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:10.304135 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a50080-a6ae-41ae-8dbc-f582608caf67 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:10.315836 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1181/2371] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 09:05:10 2022] PUT /identity/v3/projects/7edc4b9e28c646a1a746e47ccde76985/users/0756689106704e11ad17144e1a367959/roles/b870cd1acaa744e1ae1d58b9ac283821 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:05:10.328750 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bc4a75-42da-4ace-af27-b52f1d265e5b None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:10.329306 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bc4a75-42da-4ace-af27-b52f1d265e5b None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:10.329525 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bc4a75-42da-4ace-af27-b52f1d265e5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:10.340150 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:05:10.369206 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-f1bc4a75-42da-4ace-af27-b52f1d265e5b None None] MFA Rules not processed for user `0756689106704e11ad17144e1a367959`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:05:10.372058 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-f1bc4a75-42da-4ace-af27-b52f1d265e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:10.474701 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1191/2372] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:05:10 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:05:10.486147 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a50080-a6ae-41ae-8dbc-f582608caf67 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:10.487886 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-e7a50080-a6ae-41ae-8dbc-f582608caf67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:10.554032 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a50080-a6ae-41ae-8dbc-f582608caf67 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:10.557876 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3bfa47-1d7d-48aa-8025-8fa6c3908e70 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbb9ac667f04c35a63b2689df18efed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:10.558500 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3bfa47-1d7d-48aa-8025-8fa6c3908e70 service cinder] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:10.558778 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3bfa47-1d7d-48aa-8025-8fa6c3908e70 service cinder] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:10.559051 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3bfa47-1d7d-48aa-8025-8fa6c3908e70 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:10.570512 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3bfa47-1d7d-48aa-8025-8fa6c3908e70 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:10.571974 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3bfa47-1d7d-48aa-8025-8fa6c3908e70 service cinder] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:10.582684 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1182/2373] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:05:10 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:05:10.642706 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bc4a75-42da-4ace-af27-b52f1d265e5b None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:10.644390 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-f1bc4a75-42da-4ace-af27-b52f1d265e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:10.657422 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bc4a75-42da-4ace-af27-b52f1d265e5b None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:10.661950 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0217413c-4d15-41ea-a527-5478e93eee2e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbb9ac667f04c35a63b2689df18efed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:10.662622 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0217413c-4d15-41ea-a527-5478e93eee2e service cinder] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:10.662940 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0217413c-4d15-41ea-a527-5478e93eee2e service cinder] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:10.663268 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0217413c-4d15-41ea-a527-5478e93eee2e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:10.734021 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0217413c-4d15-41ea-a527-5478e93eee2e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:10.735805 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0217413c-4d15-41ea-a527-5478e93eee2e service cinder] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:10.746241 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1192/2374] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:05:10 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 09:05:16.838375 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3bfa47-1d7d-48aa-8025-8fa6c3908e70 service cinder] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:16.916146 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3bfa47-1d7d-48aa-8025-8fa6c3908e70 service cinder] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:16.920855 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6457fa97-fe7e-45fd-91ec-c0e2d370ba19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:16.921501 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6457fa97-fe7e-45fd-91ec-c0e2d370ba19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:16.921782 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6457fa97-fe7e-45fd-91ec-c0e2d370ba19 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:16.922321 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6457fa97-fe7e-45fd-91ec-c0e2d370ba19 admin admin] PATH_INFO: `/v3/users/bebf4d6902c14163844799c72563b172` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:16.923298 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6457fa97-fe7e-45fd-91ec-c0e2d370ba19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bebf4d6902c14163844799c72563b172)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:16.925302 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6457fa97-fe7e-45fd-91ec-c0e2d370ba19 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:16.941289 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:05:17.127304 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-6457fa97-fe7e-45fd-91ec-c0e2d370ba19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bebf4d6902c14163844799c72563b172'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:17.128083 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-6457fa97-fe7e-45fd-91ec-c0e2d370ba19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bebf4d6902c14163844799c72563b172'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:17.128736 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-6457fa97-fe7e-45fd-91ec-c0e2d370ba19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bebf4d6902c14163844799c72563b172'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:17.141875 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-6457fa97-fe7e-45fd-91ec-c0e2d370ba19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bebf4d6902c14163844799c72563b172'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:17.198289 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-6457fa97-fe7e-45fd-91ec-c0e2d370ba19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bebf4d6902c14163844799c72563b172'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:17.212803 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1183/2375] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 09:05:16 2022] DELETE /identity/v3/users/bebf4d6902c14163844799c72563b172 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:05:17.217552 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fab7a5-5992-4de3-b078-40313f770b0c None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:17.218192 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fab7a5-5992-4de3-b078-40313f770b0c None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:17.218192 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fab7a5-5992-4de3-b078-40313f770b0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:17.238213 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-88fab7a5-5992-4de3-b078-40313f770b0c None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:05:17.239436 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-88fab7a5-5992-4de3-b078-40313f770b0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:17.302822 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1193/2376] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:05:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:05:17.313391 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6457fa97-fe7e-45fd-91ec-c0e2d370ba19 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:17.314824 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-6457fa97-fe7e-45fd-91ec-c0e2d370ba19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:17.375611 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6457fa97-fe7e-45fd-91ec-c0e2d370ba19 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:17.379368 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811e9304-3ca9-439d-b87e-a13e1253101e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:17.379920 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811e9304-3ca9-439d-b87e-a13e1253101e service neutron] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:17.380340 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811e9304-3ca9-439d-b87e-a13e1253101e service neutron] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:17.380669 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811e9304-3ca9-439d-b87e-a13e1253101e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:17.391115 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811e9304-3ca9-439d-b87e-a13e1253101e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:17.392590 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811e9304-3ca9-439d-b87e-a13e1253101e service neutron] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:17.403090 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1184/2377] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:05:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:05:17.874891 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fab7a5-5992-4de3-b078-40313f770b0c None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:17.876348 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-88fab7a5-5992-4de3-b078-40313f770b0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:17.888948 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fab7a5-5992-4de3-b078-40313f770b0c None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:17.894045 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5f0063-4474-4155-b260-a5060edcb0b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:17.895541 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5f0063-4474-4155-b260-a5060edcb0b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:17.895541 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5f0063-4474-4155-b260-a5060edcb0b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:17.895541 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5f0063-4474-4155-b260-a5060edcb0b5 admin admin] PATH_INFO: `/v3/projects/0d0f7856ce1e49b8b70076841df4e302` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:17.896526 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5f0063-4474-4155-b260-a5060edcb0b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d0f7856ce1e49b8b70076841df4e302)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:17.898864 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5f0063-4474-4155-b260-a5060edcb0b5 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:17.926174 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:05:18.055655 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-5d5f0063-4474-4155-b260-a5060edcb0b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d0f7856ce1e49b8b70076841df4e302', 'request_id': 'req-5d5f0063-4474-4155-b260-a5060edcb0b5'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:18.061152 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-5d5f0063-4474-4155-b260-a5060edcb0b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d0f7856ce1e49b8b70076841df4e302', 'request_id': 'req-5d5f0063-4474-4155-b260-a5060edcb0b5'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:18.134562 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-5d5f0063-4474-4155-b260-a5060edcb0b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d0f7856ce1e49b8b70076841df4e302', 'request_id': 'req-5d5f0063-4474-4155-b260-a5060edcb0b5'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:18.146467 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-5d5f0063-4474-4155-b260-a5060edcb0b5 admin admin] The token cache is being invalidate because project 0d0f7856ce1e49b8b70076841df4e302 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:05:18.147252 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-5d5f0063-4474-4155-b260-a5060edcb0b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:18.147888 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-5d5f0063-4474-4155-b260-a5060edcb0b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:18.149464 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1194/2378] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 09:05:17 2022] DELETE /identity/v3/projects/0d0f7856ce1e49b8b70076841df4e302 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:05:18.153554 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811e9304-3ca9-439d-b87e-a13e1253101e service neutron] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:18.164215 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811e9304-3ca9-439d-b87e-a13e1253101e service neutron] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:18.221761 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0f0462-2d96-4fe3-94ca-b980f3fcdbea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:18.222397 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0f0462-2d96-4fe3-94ca-b980f3fcdbea admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:18.222664 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0f0462-2d96-4fe3-94ca-b980f3fcdbea admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:18.222932 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0f0462-2d96-4fe3-94ca-b980f3fcdbea admin admin] PATH_INFO: `/v3/users/0756689106704e11ad17144e1a367959` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:18.224053 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0f0462-2d96-4fe3-94ca-b980f3fcdbea admin admin] RBAC: Authorizing `identity:delete_user(user_id=0756689106704e11ad17144e1a367959)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:18.226672 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0f0462-2d96-4fe3-94ca-b980f3fcdbea admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:18.420366 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:05:18.428528 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-4a0f0462-2d96-4fe3-94ca-b980f3fcdbea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0756689106704e11ad17144e1a367959'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:18.429206 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-4a0f0462-2d96-4fe3-94ca-b980f3fcdbea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0756689106704e11ad17144e1a367959'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:18.431852 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-4a0f0462-2d96-4fe3-94ca-b980f3fcdbea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0756689106704e11ad17144e1a367959'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:18.441663 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-4a0f0462-2d96-4fe3-94ca-b980f3fcdbea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0756689106704e11ad17144e1a367959'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:18.530227 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-4a0f0462-2d96-4fe3-94ca-b980f3fcdbea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0756689106704e11ad17144e1a367959'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:18.544551 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1185/2379] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 09:05:18 2022] DELETE /identity/v3/users/0756689106704e11ad17144e1a367959 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:05:18.977141 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5f0063-4474-4155-b260-a5060edcb0b5 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:18.978700 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-5d5f0063-4474-4155-b260-a5060edcb0b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:18.993643 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5f0063-4474-4155-b260-a5060edcb0b5 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:19.062043 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0fd91e-2a65-4b37-a027-969178264d0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:19.062696 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0fd91e-2a65-4b37-a027-969178264d0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:19.062978 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0fd91e-2a65-4b37-a027-969178264d0a admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:19.063251 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0fd91e-2a65-4b37-a027-969178264d0a admin admin] PATH_INFO: `/v3/projects/7edc4b9e28c646a1a746e47ccde76985` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:19.064490 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0fd91e-2a65-4b37-a027-969178264d0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7edc4b9e28c646a1a746e47ccde76985)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:19.067818 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0fd91e-2a65-4b37-a027-969178264d0a admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:19.171252 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-fa0fd91e-2a65-4b37-a027-969178264d0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7edc4b9e28c646a1a746e47ccde76985', 'request_id': 'req-fa0fd91e-2a65-4b37-a027-969178264d0a'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:19.177357 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-fa0fd91e-2a65-4b37-a027-969178264d0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7edc4b9e28c646a1a746e47ccde76985', 'request_id': 'req-fa0fd91e-2a65-4b37-a027-969178264d0a'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:19.266850 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-fa0fd91e-2a65-4b37-a027-969178264d0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7edc4b9e28c646a1a746e47ccde76985', 'request_id': 'req-fa0fd91e-2a65-4b37-a027-969178264d0a'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:19.279482 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-fa0fd91e-2a65-4b37-a027-969178264d0a admin admin] The token cache is being invalidate because project 7edc4b9e28c646a1a746e47ccde76985 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:05:19.279768 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-fa0fd91e-2a65-4b37-a027-969178264d0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:19.280421 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-fa0fd91e-2a65-4b37-a027-969178264d0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:19.281986 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1195/2380] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 09:05:18 2022] DELETE /identity/v3/projects/7edc4b9e28c646a1a746e47ccde76985 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:05:19.287702 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29336ed-c7f5-437f-8f01-2507b936471b None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:19.288235 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29336ed-c7f5-437f-8f01-2507b936471b None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:19.288336 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29336ed-c7f5-437f-8f01-2507b936471b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:19.309332 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-b29336ed-c7f5-437f-8f01-2507b936471b None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:05:19.310586 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-b29336ed-c7f5-437f-8f01-2507b936471b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:19.381564 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1186/2381] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:05:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:05:19.389657 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f642ee-63c7-4bb9-a626-f1b6993eb7ea None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:19.389943 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f642ee-63c7-4bb9-a626-f1b6993eb7ea None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:19.390219 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f642ee-63c7-4bb9-a626-f1b6993eb7ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:19.411699 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-52f642ee-63c7-4bb9-a626-f1b6993eb7ea None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:05:19.413492 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-52f642ee-63c7-4bb9-a626-f1b6993eb7ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:19.422043 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1196/2382] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:05:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:05:19.426970 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29336ed-c7f5-437f-8f01-2507b936471b None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:19.441883 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29336ed-c7f5-437f-8f01-2507b936471b None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:19.447682 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48059818-4994-4584-bca6-ec590f2d42d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:19.448444 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48059818-4994-4584-bca6-ec590f2d42d2 admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:19.448763 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48059818-4994-4584-bca6-ec590f2d42d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:19.449121 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48059818-4994-4584-bca6-ec590f2d42d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:19.449806 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48059818-4994-4584-bca6-ec590f2d42d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:05:19.450071 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48059818-4994-4584-bca6-ec590f2d42d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:19.451683 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48059818-4994-4584-bca6-ec590f2d42d2 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:19.465280 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1187/2383] 10.222.0.55 () {58 vars in 1153 bytes} [Wed Aug 24 09:05:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:05:19.470493 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f642ee-63c7-4bb9-a626-f1b6993eb7ea None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:19.484378 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f642ee-63c7-4bb9-a626-f1b6993eb7ea None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:19.489653 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f00807b-7e0f-413e-9d1c-cc66684e1038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:19.490294 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f00807b-7e0f-413e-9d1c-cc66684e1038 admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:19.490622 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f00807b-7e0f-413e-9d1c-cc66684e1038 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:19.490892 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f00807b-7e0f-413e-9d1c-cc66684e1038 admin admin] PATH_INFO: `/v3/projects` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:19.491530 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f00807b-7e0f-413e-9d1c-cc66684e1038 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:19.493591 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f00807b-7e0f-413e-9d1c-cc66684e1038 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:19.618134 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1197/2384] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 09:05:19 2022] POST /identity/v3/projects => generated 379 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:05:19.619767 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48059818-4994-4584-bca6-ec590f2d42d2 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:19.630615 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48059818-4994-4584-bca6-ec590f2d42d2 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:19.689449 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e06600b-cb9d-40f6-aca8-a08bf5fcad30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:19.690065 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e06600b-cb9d-40f6-aca8-a08bf5fcad30 admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:19.690342 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e06600b-cb9d-40f6-aca8-a08bf5fcad30 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:19.690615 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e06600b-cb9d-40f6-aca8-a08bf5fcad30 admin admin] PATH_INFO: `/v3/users` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:19.691281 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e06600b-cb9d-40f6-aca8-a08bf5fcad30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:19.693280 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e06600b-cb9d-40f6-aca8-a08bf5fcad30 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:19.753472 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1188/2385] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 09:05:19 2022] POST /identity/v3/users => generated 290 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:05:19.757818 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f00807b-7e0f-413e-9d1c-cc66684e1038 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:19.768861 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f00807b-7e0f-413e-9d1c-cc66684e1038 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:19.774504 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0de882-473a-4da4-820e-b5b7ec442a30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:19.775142 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0de882-473a-4da4-820e-b5b7ec442a30 admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:19.775422 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0de882-473a-4da4-820e-b5b7ec442a30 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:19.775694 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0de882-473a-4da4-820e-b5b7ec442a30 admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:19.776366 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0de882-473a-4da4-820e-b5b7ec442a30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:19.778072 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0de882-473a-4da4-820e-b5b7ec442a30 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:19.789945 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1198/2386] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:05:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 09:05:19.794034 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e06600b-cb9d-40f6-aca8-a08bf5fcad30 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:19.805210 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e06600b-cb9d-40f6-aca8-a08bf5fcad30 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:19.810748 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc654e8-036c-40a4-9ef2-e8d67229b22c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:19.812010 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc654e8-036c-40a4-9ef2-e8d67229b22c admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:19.812487 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc654e8-036c-40a4-9ef2-e8d67229b22c admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:19.812992 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc654e8-036c-40a4-9ef2-e8d67229b22c admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:19.814506 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc654e8-036c-40a4-9ef2-e8d67229b22c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:19.817449 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc654e8-036c-40a4-9ef2-e8d67229b22c admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:19.838894 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1189/2387] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:05:19 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:05:19.843794 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0de882-473a-4da4-820e-b5b7ec442a30 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:19.849195 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:05:19.854885 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0de882-473a-4da4-820e-b5b7ec442a30 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:19.859737 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebadf7fb-c048-470c-a016-623b105e70dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:19.860408 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebadf7fb-c048-470c-a016-623b105e70dd admin admin] REQUEST_METHOD: `PUT` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:19.870211 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebadf7fb-c048-470c-a016-623b105e70dd admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:19.870211 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebadf7fb-c048-470c-a016-623b105e70dd admin admin] PATH_INFO: `/v3/projects/7b80384b5b634d5d8ce01d5cf5d54bc7/users/de3a02ffc1a14d0d9a59183391de77d3/roles/b674fd0a592e4e0cada811bba5ceea4d` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:19.879314 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebadf7fb-c048-470c-a016-623b105e70dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b80384b5b634d5d8ce01d5cf5d54bc7, user_id=de3a02ffc1a14d0d9a59183391de77d3, role_id=b674fd0a592e4e0cada811bba5ceea4d)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:19.882779 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebadf7fb-c048-470c-a016-623b105e70dd admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:19.947640 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1199/2388] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 09:05:19 2022] PUT /identity/v3/projects/7b80384b5b634d5d8ce01d5cf5d54bc7/users/de3a02ffc1a14d0d9a59183391de77d3/roles/b674fd0a592e4e0cada811bba5ceea4d => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:05:19.958209 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093d34a4-354c-4411-8cc7-986e8a7d4439 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:19.958487 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093d34a4-354c-4411-8cc7-986e8a7d4439 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:19.958759 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093d34a4-354c-4411-8cc7-986e8a7d4439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:19.987920 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-093d34a4-354c-4411-8cc7-986e8a7d4439 None None] MFA Rules not processed for user `de3a02ffc1a14d0d9a59183391de77d3`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:05:19.989194 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-093d34a4-354c-4411-8cc7-986e8a7d4439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:19.998547 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:05:20.059351 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1190/2389] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:05:19 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:05:20.063840 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebadf7fb-c048-470c-a016-623b105e70dd admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:20.133630 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebadf7fb-c048-470c-a016-623b105e70dd admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:20.138583 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c915fbb-db60-4267-96db-acd622ac0fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:20.139206 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c915fbb-db60-4267-96db-acd622ac0fdd admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:20.139495 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c915fbb-db60-4267-96db-acd622ac0fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:20.139771 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c915fbb-db60-4267-96db-acd622ac0fdd admin admin] PATH_INFO: `/v3/projects` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:20.140426 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c915fbb-db60-4267-96db-acd622ac0fdd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:20.142377 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c915fbb-db60-4267-96db-acd622ac0fdd admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:20.246268 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1200/2390] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 09:05:20 2022] POST /identity/v3/projects => generated 375 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:05:20.250563 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093d34a4-354c-4411-8cc7-986e8a7d4439 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:20.262645 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093d34a4-354c-4411-8cc7-986e8a7d4439 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:20.321544 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d8f94f-0727-4d32-a037-3efb11beddb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:20.322186 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d8f94f-0727-4d32-a037-3efb11beddb8 admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:20.322336 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d8f94f-0727-4d32-a037-3efb11beddb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:20.322611 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d8f94f-0727-4d32-a037-3efb11beddb8 admin admin] PATH_INFO: `/v3/users` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:20.323249 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d8f94f-0727-4d32-a037-3efb11beddb8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:20.325274 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d8f94f-0727-4d32-a037-3efb11beddb8 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:20.421429 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1191/2391] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 09:05:20 2022] POST /identity/v3/users => generated 288 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:05:20.425412 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c915fbb-db60-4267-96db-acd622ac0fdd admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:20.436120 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c915fbb-db60-4267-96db-acd622ac0fdd admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:20.440908 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8adcc2-db31-443f-b9af-02a2819c2b30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:20.441536 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8adcc2-db31-443f-b9af-02a2819c2b30 admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:20.441886 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8adcc2-db31-443f-b9af-02a2819c2b30 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:20.442167 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8adcc2-db31-443f-b9af-02a2819c2b30 admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:20.442811 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8adcc2-db31-443f-b9af-02a2819c2b30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:20.444990 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8adcc2-db31-443f-b9af-02a2819c2b30 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:20.455586 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1201/2392] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:05:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:05:20.459659 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d8f94f-0727-4d32-a037-3efb11beddb8 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:20.470260 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d8f94f-0727-4d32-a037-3efb11beddb8 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:20.474863 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab92309-7824-4587-ba02-6d6d9fd31e66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:20.475458 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab92309-7824-4587-ba02-6d6d9fd31e66 admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:20.475741 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab92309-7824-4587-ba02-6d6d9fd31e66 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:20.476018 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab92309-7824-4587-ba02-6d6d9fd31e66 admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:20.476667 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab92309-7824-4587-ba02-6d6d9fd31e66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:20.478244 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab92309-7824-4587-ba02-6d6d9fd31e66 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:20.488326 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1192/2393] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:05:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:05:20.493295 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8adcc2-db31-443f-b9af-02a2819c2b30 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:20.503413 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8adcc2-db31-443f-b9af-02a2819c2b30 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:20.508944 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705a4ba7-9fbd-446a-98d4-c850b82e390b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:20.509612 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705a4ba7-9fbd-446a-98d4-c850b82e390b admin admin] REQUEST_METHOD: `PUT` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:20.509891 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705a4ba7-9fbd-446a-98d4-c850b82e390b admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:20.510162 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705a4ba7-9fbd-446a-98d4-c850b82e390b admin admin] PATH_INFO: `/v3/projects/273f373c91274e118aca93c43859d98c/users/29a11f44693744a8bde30039c5e8b679/roles/b674fd0a592e4e0cada811bba5ceea4d` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:20.525676 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705a4ba7-9fbd-446a-98d4-c850b82e390b admin admin] RBAC: Authorizing `identity:create_grant(project_id=273f373c91274e118aca93c43859d98c, user_id=29a11f44693744a8bde30039c5e8b679, role_id=b674fd0a592e4e0cada811bba5ceea4d)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:20.528368 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705a4ba7-9fbd-446a-98d4-c850b82e390b admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:20.607078 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1202/2394] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 09:05:20 2022] PUT /identity/v3/projects/273f373c91274e118aca93c43859d98c/users/29a11f44693744a8bde30039c5e8b679/roles/b674fd0a592e4e0cada811bba5ceea4d => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:05:20.619434 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f77897e-c13e-45a4-ad88-e150f6154aa5 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:20.619727 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f77897e-c13e-45a4-ad88-e150f6154aa5 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:20.620016 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f77897e-c13e-45a4-ad88-e150f6154aa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:20.654639 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-5f77897e-c13e-45a4-ad88-e150f6154aa5 None None] MFA Rules not processed for user `29a11f44693744a8bde30039c5e8b679`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:05:20.655902 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-5f77897e-c13e-45a4-ad88-e150f6154aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:20.727072 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1193/2395] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:05:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:05:20.731858 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705a4ba7-9fbd-446a-98d4-c850b82e390b admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:20.797508 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705a4ba7-9fbd-446a-98d4-c850b82e390b admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:20.801909 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2118e6-dc88-42ae-949a-0bfedc3416af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:20.802521 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2118e6-dc88-42ae-949a-0bfedc3416af admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:20.803474 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2118e6-dc88-42ae-949a-0bfedc3416af admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:20.803758 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2118e6-dc88-42ae-949a-0bfedc3416af admin admin] PATH_INFO: `/v3/projects` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:20.804391 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2118e6-dc88-42ae-949a-0bfedc3416af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:20.806364 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2118e6-dc88-42ae-949a-0bfedc3416af admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:20.908395 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1203/2396] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 09:05:20 2022] POST /identity/v3/projects => generated 377 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:05:20.913049 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f77897e-c13e-45a4-ad88-e150f6154aa5 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:20.923762 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f77897e-c13e-45a4-ad88-e150f6154aa5 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:20.936835 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:05:20.982521 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a204b83c-2ea6-4e15-9bc1-c228f4250fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:20.983123 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a204b83c-2ea6-4e15-9bc1-c228f4250fd5 admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:20.983404 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a204b83c-2ea6-4e15-9bc1-c228f4250fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:20.983677 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a204b83c-2ea6-4e15-9bc1-c228f4250fd5 admin admin] PATH_INFO: `/v3/users` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:20.984297 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a204b83c-2ea6-4e15-9bc1-c228f4250fd5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:20.986287 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a204b83c-2ea6-4e15-9bc1-c228f4250fd5 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:21.060859 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1194/2397] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 09:05:20 2022] POST /identity/v3/users => generated 295 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:05:21.065322 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2118e6-dc88-42ae-949a-0bfedc3416af admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:21.076164 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2118e6-dc88-42ae-949a-0bfedc3416af admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:21.081474 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39e9f71-1fa9-47ba-a166-3c52c1581591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:21.082133 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39e9f71-1fa9-47ba-a166-3c52c1581591 admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:21.082412 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39e9f71-1fa9-47ba-a166-3c52c1581591 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:21.082686 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39e9f71-1fa9-47ba-a166-3c52c1581591 admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:21.083368 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39e9f71-1fa9-47ba-a166-3c52c1581591 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:21.085658 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39e9f71-1fa9-47ba-a166-3c52c1581591 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:21.097933 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1204/2398] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:05:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:05:21.104942 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a204b83c-2ea6-4e15-9bc1-c228f4250fd5 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:21.115948 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a204b83c-2ea6-4e15-9bc1-c228f4250fd5 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:21.120667 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4921f5-66cd-4489-b1e6-b9af4eb64eeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:21.121346 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4921f5-66cd-4489-b1e6-b9af4eb64eeb admin admin] REQUEST_METHOD: `PUT` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:21.121642 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4921f5-66cd-4489-b1e6-b9af4eb64eeb admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:21.121945 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4921f5-66cd-4489-b1e6-b9af4eb64eeb admin admin] PATH_INFO: `/v3/projects/c9b23bbbcfd3408e9f10fff1fcaf23c5/users/e6bcf9f3dafb4514a9ef9bdfa54cbdf9/roles/b870cd1acaa744e1ae1d58b9ac283821` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:21.139252 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4921f5-66cd-4489-b1e6-b9af4eb64eeb admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9b23bbbcfd3408e9f10fff1fcaf23c5, user_id=e6bcf9f3dafb4514a9ef9bdfa54cbdf9, role_id=b870cd1acaa744e1ae1d58b9ac283821)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:21.141977 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4921f5-66cd-4489-b1e6-b9af4eb64eeb admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:21.255120 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1195/2399] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 09:05:21 2022] PUT /identity/v3/projects/c9b23bbbcfd3408e9f10fff1fcaf23c5/users/e6bcf9f3dafb4514a9ef9bdfa54cbdf9/roles/b870cd1acaa744e1ae1d58b9ac283821 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:05:21.259507 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39e9f71-1fa9-47ba-a166-3c52c1581591 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:21.264753 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:05:21.270542 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39e9f71-1fa9-47ba-a166-3c52c1581591 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:21.334820 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb753c1-0aa0-48b4-a3bc-c82b2f2e72d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:21.335487 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb753c1-0aa0-48b4-a3bc-c82b2f2e72d7 admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:21.336203 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb753c1-0aa0-48b4-a3bc-c82b2f2e72d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:21.337239 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb753c1-0aa0-48b4-a3bc-c82b2f2e72d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:21.337420 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb753c1-0aa0-48b4-a3bc-c82b2f2e72d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:21.339580 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb753c1-0aa0-48b4-a3bc-c82b2f2e72d7 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:21.350124 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1205/2400] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:05:21 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:05:21.354200 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4921f5-66cd-4489-b1e6-b9af4eb64eeb admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:21.364851 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4921f5-66cd-4489-b1e6-b9af4eb64eeb admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:21.369749 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8687538-69ba-4819-97b1-153c30f5861f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:21.370409 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8687538-69ba-4819-97b1-153c30f5861f admin admin] REQUEST_METHOD: `PUT` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:21.370758 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8687538-69ba-4819-97b1-153c30f5861f admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:21.371036 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8687538-69ba-4819-97b1-153c30f5861f admin admin] PATH_INFO: `/v3/projects/c9b23bbbcfd3408e9f10fff1fcaf23c5/users/e6bcf9f3dafb4514a9ef9bdfa54cbdf9/roles/b870cd1acaa744e1ae1d58b9ac283821` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:21.372992 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8687538-69ba-4819-97b1-153c30f5861f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9b23bbbcfd3408e9f10fff1fcaf23c5, user_id=e6bcf9f3dafb4514a9ef9bdfa54cbdf9, role_id=b870cd1acaa744e1ae1d58b9ac283821)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:21.375719 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8687538-69ba-4819-97b1-153c30f5861f admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:21.386993 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1196/2401] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 09:05:21 2022] PUT /identity/v3/projects/c9b23bbbcfd3408e9f10fff1fcaf23c5/users/e6bcf9f3dafb4514a9ef9bdfa54cbdf9/roles/b870cd1acaa744e1ae1d58b9ac283821 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:05:21.397854 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90622f6c-cfeb-44c7-92e7-e36e7f0d8d70 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:21.398137 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90622f6c-cfeb-44c7-92e7-e36e7f0d8d70 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:21.398407 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90622f6c-cfeb-44c7-92e7-e36e7f0d8d70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:21.430812 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-90622f6c-cfeb-44c7-92e7-e36e7f0d8d70 None None] MFA Rules not processed for user `e6bcf9f3dafb4514a9ef9bdfa54cbdf9`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:05:21.432052 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-90622f6c-cfeb-44c7-92e7-e36e7f0d8d70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:21.519995 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1206/2402] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:05:21 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:05:21.532205 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8687538-69ba-4819-97b1-153c30f5861f admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:21.534322 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-d8687538-69ba-4819-97b1-153c30f5861f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:21.594679 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8687538-69ba-4819-97b1-153c30f5861f admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:21.598416 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa82cd6-ccc6-42c6-941d-aaa81fd0d7f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbb9ac667f04c35a63b2689df18efed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:21.599031 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa82cd6-ccc6-42c6-941d-aaa81fd0d7f4 service cinder] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:21.599316 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa82cd6-ccc6-42c6-941d-aaa81fd0d7f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:21.599619 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa82cd6-ccc6-42c6-941d-aaa81fd0d7f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:21.611828 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa82cd6-ccc6-42c6-941d-aaa81fd0d7f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:21.613504 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa82cd6-ccc6-42c6-941d-aaa81fd0d7f4 service cinder] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:21.624323 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1197/2403] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:05:21 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:05:21.727672 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90622f6c-cfeb-44c7-92e7-e36e7f0d8d70 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:21.729135 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-90622f6c-cfeb-44c7-92e7-e36e7f0d8d70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:21.742226 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90622f6c-cfeb-44c7-92e7-e36e7f0d8d70 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:21.745878 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317bf90a-fa21-4042-881d-d6d78887e71f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbb9ac667f04c35a63b2689df18efed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:21.746484 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317bf90a-fa21-4042-881d-d6d78887e71f service cinder] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:21.746764 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317bf90a-fa21-4042-881d-d6d78887e71f service cinder] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:21.747037 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317bf90a-fa21-4042-881d-d6d78887e71f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:21.805482 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317bf90a-fa21-4042-881d-d6d78887e71f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:21.806936 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317bf90a-fa21-4042-881d-d6d78887e71f service cinder] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:21.817409 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1207/2404] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:05:21 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:05:25.523529 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c691fc-d058-4546-8039-1db827bd00ad None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:25.524022 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c691fc-d058-4546-8039-1db827bd00ad None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:25.524260 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c691fc-d058-4546-8039-1db827bd00ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:25.554446 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-f7c691fc-d058-4546-8039-1db827bd00ad None None] MFA Rules not processed for user `f652a368b1cc42ae8b6f6c7a6dff6f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:05:25.555652 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-f7c691fc-d058-4546-8039-1db827bd00ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:25.619891 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1198/2405] 10.222.0.55 () {58 vars in 967 bytes} [Wed Aug 24 09:05:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:05:25.756203 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317bf90a-fa21-4042-881d-d6d78887e71f service cinder] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:25.758307 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-317bf90a-fa21-4042-881d-d6d78887e71f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:25.795880 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:05:25.841948 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317bf90a-fa21-4042-881d-d6d78887e71f service cinder] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:25.847598 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe891b1-ea2f-44f7-9fca-cccef0b515a4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0ebb5ccabef45378f0c1da69721e051', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:25.848219 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe891b1-ea2f-44f7-9fca-cccef0b515a4 service swift] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:25.848501 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe891b1-ea2f-44f7-9fca-cccef0b515a4 service swift] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:25.848806 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe891b1-ea2f-44f7-9fca-cccef0b515a4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:25.859214 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe891b1-ea2f-44f7-9fca-cccef0b515a4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:25.860888 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe891b1-ea2f-44f7-9fca-cccef0b515a4 service swift] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:25.870600 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1208/2406] 10.222.0.55 () {60 vars in 1384 bytes} [Wed Aug 24 09:05:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 09:05:27.162373 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c691fc-d058-4546-8039-1db827bd00ad None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:27.172697 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:05:27.231932 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c691fc-d058-4546-8039-1db827bd00ad None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:27.236877 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf13490-f627-489a-b462-944621fa359b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:27.237500 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf13490-f627-489a-b462-944621fa359b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:27.237774 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf13490-f627-489a-b462-944621fa359b admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:27.238045 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf13490-f627-489a-b462-944621fa359b admin admin] PATH_INFO: `/v3/users/de3a02ffc1a14d0d9a59183391de77d3` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:27.239218 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf13490-f627-489a-b462-944621fa359b admin admin] RBAC: Authorizing `identity:delete_user(user_id=de3a02ffc1a14d0d9a59183391de77d3)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:27.241717 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf13490-f627-489a-b462-944621fa359b admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:27.431015 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-7bf13490-f627-489a-b462-944621fa359b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de3a02ffc1a14d0d9a59183391de77d3'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:27.431724 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-7bf13490-f627-489a-b462-944621fa359b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de3a02ffc1a14d0d9a59183391de77d3'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:27.432341 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-7bf13490-f627-489a-b462-944621fa359b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de3a02ffc1a14d0d9a59183391de77d3'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:27.446502 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-7bf13490-f627-489a-b462-944621fa359b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de3a02ffc1a14d0d9a59183391de77d3'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:27.518066 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-7bf13490-f627-489a-b462-944621fa359b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de3a02ffc1a14d0d9a59183391de77d3'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:27.531945 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1199/2407] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 09:05:27 2022] DELETE /identity/v3/users/de3a02ffc1a14d0d9a59183391de77d3 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:05:27.542052 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9fe18b-1c76-404e-9487-ed29b352a58e None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:27.542606 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9fe18b-1c76-404e-9487-ed29b352a58e None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:27.542606 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9fe18b-1c76-404e-9487-ed29b352a58e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:27.563259 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-5b9fe18b-1c76-404e-9487-ed29b352a58e None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:05:27.564480 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-5b9fe18b-1c76-404e-9487-ed29b352a58e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:27.635524 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1209/2408] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:05:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:05:27.645938 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf13490-f627-489a-b462-944621fa359b admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:27.647636 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-7bf13490-f627-489a-b462-944621fa359b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:27.712512 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf13490-f627-489a-b462-944621fa359b admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:27.716441 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca28d5e-0916-40ea-9830-90d29002d574 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:27.717110 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca28d5e-0916-40ea-9830-90d29002d574 service neutron] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:27.717390 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca28d5e-0916-40ea-9830-90d29002d574 service neutron] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:27.717660 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca28d5e-0916-40ea-9830-90d29002d574 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:27.728258 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca28d5e-0916-40ea-9830-90d29002d574 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:27.729733 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca28d5e-0916-40ea-9830-90d29002d574 service neutron] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:27.739804 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1200/2409] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:05:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:05:28.366197 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9fe18b-1c76-404e-9487-ed29b352a58e None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:28.370480 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-5b9fe18b-1c76-404e-9487-ed29b352a58e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:28.383702 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9fe18b-1c76-404e-9487-ed29b352a58e None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:28.393554 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4208f848-c387-4225-8ff1-10976245bd18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:28.395396 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4208f848-c387-4225-8ff1-10976245bd18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:28.396199 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4208f848-c387-4225-8ff1-10976245bd18 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:28.396737 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4208f848-c387-4225-8ff1-10976245bd18 admin admin] PATH_INFO: `/v3/projects/7b80384b5b634d5d8ce01d5cf5d54bc7` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:28.406176 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4208f848-c387-4225-8ff1-10976245bd18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b80384b5b634d5d8ce01d5cf5d54bc7)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:28.406176 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4208f848-c387-4225-8ff1-10976245bd18 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:28.535892 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-4208f848-c387-4225-8ff1-10976245bd18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b80384b5b634d5d8ce01d5cf5d54bc7', 'request_id': 'req-4208f848-c387-4225-8ff1-10976245bd18'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:28.547304 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-4208f848-c387-4225-8ff1-10976245bd18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b80384b5b634d5d8ce01d5cf5d54bc7', 'request_id': 'req-4208f848-c387-4225-8ff1-10976245bd18'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:28.620886 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-4208f848-c387-4225-8ff1-10976245bd18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b80384b5b634d5d8ce01d5cf5d54bc7', 'request_id': 'req-4208f848-c387-4225-8ff1-10976245bd18'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:28.634357 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-4208f848-c387-4225-8ff1-10976245bd18 admin admin] The token cache is being invalidate because project 7b80384b5b634d5d8ce01d5cf5d54bc7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:05:28.634642 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-4208f848-c387-4225-8ff1-10976245bd18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:28.635305 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-4208f848-c387-4225-8ff1-10976245bd18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:28.636903 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1210/2410] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 09:05:28 2022] DELETE /identity/v3/projects/7b80384b5b634d5d8ce01d5cf5d54bc7 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:05:28.641019 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca28d5e-0916-40ea-9830-90d29002d574 service neutron] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:28.651201 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca28d5e-0916-40ea-9830-90d29002d574 service neutron] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:28.683167 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:05:28.713157 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139d6a8e-d5c0-47fc-91b5-0113f4d65fa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:28.714898 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d6a8e-d5c0-47fc-91b5-0113f4d65fa4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:28.714898 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d6a8e-d5c0-47fc-91b5-0113f4d65fa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:28.714898 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d6a8e-d5c0-47fc-91b5-0113f4d65fa4 admin admin] PATH_INFO: `/v3/users/29a11f44693744a8bde30039c5e8b679` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:28.715692 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139d6a8e-d5c0-47fc-91b5-0113f4d65fa4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=29a11f44693744a8bde30039c5e8b679)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:28.718086 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139d6a8e-d5c0-47fc-91b5-0113f4d65fa4 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:28.930691 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-139d6a8e-d5c0-47fc-91b5-0113f4d65fa4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29a11f44693744a8bde30039c5e8b679'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:28.931578 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-139d6a8e-d5c0-47fc-91b5-0113f4d65fa4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29a11f44693744a8bde30039c5e8b679'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:28.932430 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-139d6a8e-d5c0-47fc-91b5-0113f4d65fa4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29a11f44693744a8bde30039c5e8b679'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:28.948428 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-139d6a8e-d5c0-47fc-91b5-0113f4d65fa4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29a11f44693744a8bde30039c5e8b679'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:29.009416 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-139d6a8e-d5c0-47fc-91b5-0113f4d65fa4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29a11f44693744a8bde30039c5e8b679'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:29.024880 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1201/2411] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 09:05:28 2022] DELETE /identity/v3/users/29a11f44693744a8bde30039c5e8b679 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:05:29.714385 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4208f848-c387-4225-8ff1-10976245bd18 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:29.715680 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-4208f848-c387-4225-8ff1-10976245bd18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:29.731561 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4208f848-c387-4225-8ff1-10976245bd18 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:29.782506 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:05:29.808048 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1be2355-6a46-4f0f-b30e-2c1926e82a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:29.808806 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be2355-6a46-4f0f-b30e-2c1926e82a29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:29.809085 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be2355-6a46-4f0f-b30e-2c1926e82a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:29.809356 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be2355-6a46-4f0f-b30e-2c1926e82a29 admin admin] PATH_INFO: `/v3/projects/273f373c91274e118aca93c43859d98c` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:29.810516 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1be2355-6a46-4f0f-b30e-2c1926e82a29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=273f373c91274e118aca93c43859d98c)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:29.812763 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1be2355-6a46-4f0f-b30e-2c1926e82a29 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:29.941157 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-e1be2355-6a46-4f0f-b30e-2c1926e82a29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '273f373c91274e118aca93c43859d98c', 'request_id': 'req-e1be2355-6a46-4f0f-b30e-2c1926e82a29'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:29.952201 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-e1be2355-6a46-4f0f-b30e-2c1926e82a29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '273f373c91274e118aca93c43859d98c', 'request_id': 'req-e1be2355-6a46-4f0f-b30e-2c1926e82a29'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:30.043991 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-e1be2355-6a46-4f0f-b30e-2c1926e82a29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '273f373c91274e118aca93c43859d98c', 'request_id': 'req-e1be2355-6a46-4f0f-b30e-2c1926e82a29'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:30.059644 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-e1be2355-6a46-4f0f-b30e-2c1926e82a29 admin admin] The token cache is being invalidate because project 273f373c91274e118aca93c43859d98c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:05:30.059946 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-e1be2355-6a46-4f0f-b30e-2c1926e82a29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:30.060681 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-e1be2355-6a46-4f0f-b30e-2c1926e82a29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:30.062287 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1211/2412] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 09:05:29 2022] DELETE /identity/v3/projects/273f373c91274e118aca93c43859d98c => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:05:30.066401 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139d6a8e-d5c0-47fc-91b5-0113f4d65fa4 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:30.068109 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-139d6a8e-d5c0-47fc-91b5-0113f4d65fa4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:30.082765 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139d6a8e-d5c0-47fc-91b5-0113f4d65fa4 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:30.148236 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155230ab-bd92-4b86-a781-5b50726aba0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:30.148997 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155230ab-bd92-4b86-a781-5b50726aba0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:30.153796 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155230ab-bd92-4b86-a781-5b50726aba0d admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:30.153796 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155230ab-bd92-4b86-a781-5b50726aba0d admin admin] PATH_INFO: `/v3/users/e6bcf9f3dafb4514a9ef9bdfa54cbdf9` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:30.153796 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155230ab-bd92-4b86-a781-5b50726aba0d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6bcf9f3dafb4514a9ef9bdfa54cbdf9)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:30.153796 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155230ab-bd92-4b86-a781-5b50726aba0d admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:30.399961 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-155230ab-bd92-4b86-a781-5b50726aba0d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6bcf9f3dafb4514a9ef9bdfa54cbdf9'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:30.400682 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-155230ab-bd92-4b86-a781-5b50726aba0d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6bcf9f3dafb4514a9ef9bdfa54cbdf9'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:30.401344 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-155230ab-bd92-4b86-a781-5b50726aba0d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6bcf9f3dafb4514a9ef9bdfa54cbdf9'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:30.407050 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:05:30.416906 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-155230ab-bd92-4b86-a781-5b50726aba0d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6bcf9f3dafb4514a9ef9bdfa54cbdf9'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:30.542196 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-155230ab-bd92-4b86-a781-5b50726aba0d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6bcf9f3dafb4514a9ef9bdfa54cbdf9'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:30.562179 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1202/2413] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 09:05:30 2022] DELETE /identity/v3/users/e6bcf9f3dafb4514a9ef9bdfa54cbdf9 => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:05:31.151474 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1be2355-6a46-4f0f-b30e-2c1926e82a29 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:31.153094 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-e1be2355-6a46-4f0f-b30e-2c1926e82a29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:31.166800 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1be2355-6a46-4f0f-b30e-2c1926e82a29 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:31.231855 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155230ab-bd92-4b86-a781-5b50726aba0d admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:31.235190 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-155230ab-bd92-4b86-a781-5b50726aba0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:31.276643 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe3cce9-e544-420b-80fb-1783393a98f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:31.277256 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe3cce9-e544-420b-80fb-1783393a98f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:31.277811 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe3cce9-e544-420b-80fb-1783393a98f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:31.278096 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe3cce9-e544-420b-80fb-1783393a98f2 admin admin] PATH_INFO: `/v3/projects/c9b23bbbcfd3408e9f10fff1fcaf23c5` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:31.280182 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe3cce9-e544-420b-80fb-1783393a98f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9b23bbbcfd3408e9f10fff1fcaf23c5)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:31.283945 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe3cce9-e544-420b-80fb-1783393a98f2 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:31.327071 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155230ab-bd92-4b86-a781-5b50726aba0d admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:31.332687 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083ec824-0b6c-4731-aa3d-cc348ef77f4c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf0c99771661413aaa0f237af6b394ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:31.333336 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083ec824-0b6c-4731-aa3d-cc348ef77f4c service placement] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:31.333623 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083ec824-0b6c-4731-aa3d-cc348ef77f4c service placement] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:31.333906 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083ec824-0b6c-4731-aa3d-cc348ef77f4c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:31.336521 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-083ec824-0b6c-4731-aa3d-cc348ef77f4c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:31.349331 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083ec824-0b6c-4731-aa3d-cc348ef77f4c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:31.351069 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083ec824-0b6c-4731-aa3d-cc348ef77f4c service placement] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:31.362996 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1203/2414] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:05:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 09:05:31.437146 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-9fe3cce9-e544-420b-80fb-1783393a98f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9b23bbbcfd3408e9f10fff1fcaf23c5', 'request_id': 'req-9fe3cce9-e544-420b-80fb-1783393a98f2'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:31.443183 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-9fe3cce9-e544-420b-80fb-1783393a98f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9b23bbbcfd3408e9f10fff1fcaf23c5', 'request_id': 'req-9fe3cce9-e544-420b-80fb-1783393a98f2'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:31.518225 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-9fe3cce9-e544-420b-80fb-1783393a98f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9b23bbbcfd3408e9f10fff1fcaf23c5', 'request_id': 'req-9fe3cce9-e544-420b-80fb-1783393a98f2'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:31.523307 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:05:31.538295 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-9fe3cce9-e544-420b-80fb-1783393a98f2 admin admin] The token cache is being invalidate because project c9b23bbbcfd3408e9f10fff1fcaf23c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:05:31.538295 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-9fe3cce9-e544-420b-80fb-1783393a98f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:31.540304 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-9fe3cce9-e544-420b-80fb-1783393a98f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:31.546388 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1212/2415] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 09:05:31 2022] DELETE /identity/v3/projects/c9b23bbbcfd3408e9f10fff1fcaf23c5 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:05:31.554975 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e906e3ae-0376-4c45-addd-6ae6bbe9f392 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:31.557118 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e906e3ae-0376-4c45-addd-6ae6bbe9f392 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:31.557118 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e906e3ae-0376-4c45-addd-6ae6bbe9f392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:31.579020 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-e906e3ae-0376-4c45-addd-6ae6bbe9f392 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:05:31.580300 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-e906e3ae-0376-4c45-addd-6ae6bbe9f392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:31.674653 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1204/2416] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:05:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:05:31.684941 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220e0119-cb42-4d33-aa80-c248ee7b6afd None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:31.685312 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220e0119-cb42-4d33-aa80-c248ee7b6afd None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:31.685655 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220e0119-cb42-4d33-aa80-c248ee7b6afd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:31.707437 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-220e0119-cb42-4d33-aa80-c248ee7b6afd None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:05:31.708811 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-220e0119-cb42-4d33-aa80-c248ee7b6afd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:31.716916 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1213/2417] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:05:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:05:31.721398 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e906e3ae-0376-4c45-addd-6ae6bbe9f392 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:31.734141 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e906e3ae-0376-4c45-addd-6ae6bbe9f392 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:31.739379 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fb270e-3ecc-4279-bc85-3f5b3e67a63e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:31.740749 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fb270e-3ecc-4279-bc85-3f5b3e67a63e admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:31.741690 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fb270e-3ecc-4279-bc85-3f5b3e67a63e admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:31.741828 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fb270e-3ecc-4279-bc85-3f5b3e67a63e admin admin] PATH_INFO: `/v3/domains` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:31.742434 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fb270e-3ecc-4279-bc85-3f5b3e67a63e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:05:31.742743 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fb270e-3ecc-4279-bc85-3f5b3e67a63e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:31.744382 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fb270e-3ecc-4279-bc85-3f5b3e67a63e admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:31.757781 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1205/2418] 10.222.0.55 () {58 vars in 1153 bytes} [Wed Aug 24 09:05:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:05:31.767870 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220e0119-cb42-4d33-aa80-c248ee7b6afd None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:31.778187 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220e0119-cb42-4d33-aa80-c248ee7b6afd None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:31.784127 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d303f377-c266-48e9-8647-f4c8638c3a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:31.785297 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d303f377-c266-48e9-8647-f4c8638c3a34 admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:31.785759 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d303f377-c266-48e9-8647-f4c8638c3a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:31.786248 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d303f377-c266-48e9-8647-f4c8638c3a34 admin admin] PATH_INFO: `/v3/projects` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:31.787258 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d303f377-c266-48e9-8647-f4c8638c3a34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:31.791203 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d303f377-c266-48e9-8647-f4c8638c3a34 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:31.886914 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1214/2419] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 09:05:31 2022] POST /identity/v3/projects => generated 383 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:05:31.891605 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fb270e-3ecc-4279-bc85-3f5b3e67a63e admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:31.903752 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fb270e-3ecc-4279-bc85-3f5b3e67a63e admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:31.955766 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:05:31.970354 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d0f97f-af44-494b-8aa0-a9b435a033ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:31.971152 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d0f97f-af44-494b-8aa0-a9b435a033ee admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:31.971435 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d0f97f-af44-494b-8aa0-a9b435a033ee admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:31.971733 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d0f97f-af44-494b-8aa0-a9b435a033ee admin admin] PATH_INFO: `/v3/users` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:31.972380 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d0f97f-af44-494b-8aa0-a9b435a033ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:31.974420 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d0f97f-af44-494b-8aa0-a9b435a033ee admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:32.077511 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1206/2420] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 09:05:31 2022] POST /identity/v3/users => generated 292 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:05:32.082791 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d303f377-c266-48e9-8647-f4c8638c3a34 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:32.093323 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d303f377-c266-48e9-8647-f4c8638c3a34 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:32.099592 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838e1581-d2de-46b9-b3d2-5adc26b3e96c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:32.100230 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838e1581-d2de-46b9-b3d2-5adc26b3e96c admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:32.101241 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838e1581-d2de-46b9-b3d2-5adc26b3e96c admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:32.101520 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838e1581-d2de-46b9-b3d2-5adc26b3e96c admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:32.102177 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838e1581-d2de-46b9-b3d2-5adc26b3e96c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:32.103928 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838e1581-d2de-46b9-b3d2-5adc26b3e96c admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:32.120891 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1215/2421] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:05:32 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:05:32.121896 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d0f97f-af44-494b-8aa0-a9b435a033ee admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:32.132417 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d0f97f-af44-494b-8aa0-a9b435a033ee admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:32.137987 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e44203-fb3a-48c5-90e9-889eaa104832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:32.138606 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e44203-fb3a-48c5-90e9-889eaa104832 admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:32.138888 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e44203-fb3a-48c5-90e9-889eaa104832 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:32.139262 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e44203-fb3a-48c5-90e9-889eaa104832 admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:32.139911 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e44203-fb3a-48c5-90e9-889eaa104832 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:32.141544 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e44203-fb3a-48c5-90e9-889eaa104832 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:32.157219 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1207/2422] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:05:32 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:05:32.161673 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838e1581-d2de-46b9-b3d2-5adc26b3e96c admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:32.172912 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838e1581-d2de-46b9-b3d2-5adc26b3e96c admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:32.177673 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4118b0ac-a43f-4cd0-a03b-ef385d624cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:32.178334 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4118b0ac-a43f-4cd0-a03b-ef385d624cb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:32.178653 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4118b0ac-a43f-4cd0-a03b-ef385d624cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:32.179431 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4118b0ac-a43f-4cd0-a03b-ef385d624cb7 admin admin] PATH_INFO: `/v3/projects/84efd7e61a0b4ee8a3c8394af63c3ff0/users/03fcad2f720446899501463f77edb9f1/roles/b674fd0a592e4e0cada811bba5ceea4d` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:32.198194 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4118b0ac-a43f-4cd0-a03b-ef385d624cb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84efd7e61a0b4ee8a3c8394af63c3ff0, user_id=03fcad2f720446899501463f77edb9f1, role_id=b674fd0a592e4e0cada811bba5ceea4d)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:32.201085 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4118b0ac-a43f-4cd0-a03b-ef385d624cb7 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:32.267666 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1216/2423] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 09:05:32 2022] PUT /identity/v3/projects/84efd7e61a0b4ee8a3c8394af63c3ff0/users/03fcad2f720446899501463f77edb9f1/roles/b674fd0a592e4e0cada811bba5ceea4d => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:05:32.341799 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ced599-19e0-4a6f-b7fb-84323c1b8a2b None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:32.342346 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ced599-19e0-4a6f-b7fb-84323c1b8a2b None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:32.342346 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ced599-19e0-4a6f-b7fb-84323c1b8a2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:32.374606 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-78ced599-19e0-4a6f-b7fb-84323c1b8a2b None None] MFA Rules not processed for user `03fcad2f720446899501463f77edb9f1`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:05:32.375842 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-78ced599-19e0-4a6f-b7fb-84323c1b8a2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:32.449171 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1208/2424] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:05:32 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:05:32.464307 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4118b0ac-a43f-4cd0-a03b-ef385d624cb7 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:32.466144 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-4118b0ac-a43f-4cd0-a03b-ef385d624cb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:32.537223 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4118b0ac-a43f-4cd0-a03b-ef385d624cb7 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:32.541574 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770eb8e7-a8ff-4a24-9c18-7204b5a8101a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbb9ac667f04c35a63b2689df18efed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:32.542187 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770eb8e7-a8ff-4a24-9c18-7204b5a8101a service cinder] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:32.542464 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770eb8e7-a8ff-4a24-9c18-7204b5a8101a service cinder] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:32.542734 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770eb8e7-a8ff-4a24-9c18-7204b5a8101a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:32.551909 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770eb8e7-a8ff-4a24-9c18-7204b5a8101a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:32.553396 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770eb8e7-a8ff-4a24-9c18-7204b5a8101a service cinder] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:32.563505 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1217/2425] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:05:32 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:05:35.935738 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ced599-19e0-4a6f-b7fb-84323c1b8a2b None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:35.937991 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-78ced599-19e0-4a6f-b7fb-84323c1b8a2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:36.013913 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ced599-19e0-4a6f-b7fb-84323c1b8a2b None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:36.018514 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884dfb61-7369-48cd-9aef-420e3179c903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:36.019182 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884dfb61-7369-48cd-9aef-420e3179c903 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:36.019444 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884dfb61-7369-48cd-9aef-420e3179c903 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:36.019719 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884dfb61-7369-48cd-9aef-420e3179c903 admin admin] PATH_INFO: `/v3/users/095ae2b97081460da5d3da2366a604c3` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:36.020927 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884dfb61-7369-48cd-9aef-420e3179c903 admin admin] RBAC: Authorizing `identity:delete_user(user_id=095ae2b97081460da5d3da2366a604c3)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:36.022850 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884dfb61-7369-48cd-9aef-420e3179c903 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:36.266534 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-884dfb61-7369-48cd-9aef-420e3179c903 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '095ae2b97081460da5d3da2366a604c3'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:36.267175 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-884dfb61-7369-48cd-9aef-420e3179c903 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '095ae2b97081460da5d3da2366a604c3'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:36.267781 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-884dfb61-7369-48cd-9aef-420e3179c903 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '095ae2b97081460da5d3da2366a604c3'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:36.275246 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:05:36.280529 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-884dfb61-7369-48cd-9aef-420e3179c903 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '095ae2b97081460da5d3da2366a604c3'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:36.342609 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-884dfb61-7369-48cd-9aef-420e3179c903 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '095ae2b97081460da5d3da2366a604c3'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:36.357638 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1209/2426] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 09:05:35 2022] DELETE /identity/v3/users/095ae2b97081460da5d3da2366a604c3 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:05:36.685671 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770eb8e7-a8ff-4a24-9c18-7204b5a8101a service cinder] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:36.698945 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770eb8e7-a8ff-4a24-9c18-7204b5a8101a service cinder] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:36.787206 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029ed811-c16a-4c6f-a9f9-9641fcb75a63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:36.787847 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029ed811-c16a-4c6f-a9f9-9641fcb75a63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:36.788125 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029ed811-c16a-4c6f-a9f9-9641fcb75a63 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:36.788406 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029ed811-c16a-4c6f-a9f9-9641fcb75a63 admin admin] PATH_INFO: `/v3/projects/eccd422b1e434b56ba0e0a31aebebd98` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:36.790099 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029ed811-c16a-4c6f-a9f9-9641fcb75a63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eccd422b1e434b56ba0e0a31aebebd98)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:36.792058 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029ed811-c16a-4c6f-a9f9-9641fcb75a63 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:36.876882 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884dfb61-7369-48cd-9aef-420e3179c903 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:36.878368 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-884dfb61-7369-48cd-9aef-420e3179c903 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:36.892057 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884dfb61-7369-48cd-9aef-420e3179c903 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:36.896911 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8c65eb-b4ea-416b-8159-35abe098b4e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:36.897553 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8c65eb-b4ea-416b-8159-35abe098b4e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:36.897835 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8c65eb-b4ea-416b-8159-35abe098b4e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:36.898111 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8c65eb-b4ea-416b-8159-35abe098b4e1 admin admin] PATH_INFO: `/v3/users/03fcad2f720446899501463f77edb9f1` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:36.899946 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8c65eb-b4ea-416b-8159-35abe098b4e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=03fcad2f720446899501463f77edb9f1)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:36.901935 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8c65eb-b4ea-416b-8159-35abe098b4e1 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:36.904202 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:05:36.921827 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-029ed811-c16a-4c6f-a9f9-9641fcb75a63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eccd422b1e434b56ba0e0a31aebebd98', 'request_id': 'req-029ed811-c16a-4c6f-a9f9-9641fcb75a63'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:36.929078 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-029ed811-c16a-4c6f-a9f9-9641fcb75a63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eccd422b1e434b56ba0e0a31aebebd98', 'request_id': 'req-029ed811-c16a-4c6f-a9f9-9641fcb75a63'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:37.026595 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-029ed811-c16a-4c6f-a9f9-9641fcb75a63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eccd422b1e434b56ba0e0a31aebebd98', 'request_id': 'req-029ed811-c16a-4c6f-a9f9-9641fcb75a63'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:37.047157 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-029ed811-c16a-4c6f-a9f9-9641fcb75a63 admin admin] The token cache is being invalidate because project eccd422b1e434b56ba0e0a31aebebd98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:05:37.047661 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-029ed811-c16a-4c6f-a9f9-9641fcb75a63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:37.048840 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-029ed811-c16a-4c6f-a9f9-9641fcb75a63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:37.051413 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1218/2427] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 09:05:36 2022] DELETE /identity/v3/projects/eccd422b1e434b56ba0e0a31aebebd98 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:05:37.055875 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029ed811-c16a-4c6f-a9f9-9641fcb75a63 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:37.057359 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-029ed811-c16a-4c6f-a9f9-9641fcb75a63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:37.128182 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-bb8c65eb-b4ea-416b-8159-35abe098b4e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03fcad2f720446899501463f77edb9f1'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:37.129295 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-bb8c65eb-b4ea-416b-8159-35abe098b4e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03fcad2f720446899501463f77edb9f1'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:37.130252 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-bb8c65eb-b4ea-416b-8159-35abe098b4e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03fcad2f720446899501463f77edb9f1'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:37.134355 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029ed811-c16a-4c6f-a9f9-9641fcb75a63 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:37.139397 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919e86bf-77a2-4f2a-ad72-2470a973df67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:37.140073 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919e86bf-77a2-4f2a-ad72-2470a973df67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:37.140343 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919e86bf-77a2-4f2a-ad72-2470a973df67 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:37.140647 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919e86bf-77a2-4f2a-ad72-2470a973df67 admin admin] PATH_INFO: `/v3/users/6eb4015e1c38446daeeeb884e5495870` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:37.141878 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919e86bf-77a2-4f2a-ad72-2470a973df67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6eb4015e1c38446daeeeb884e5495870)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:37.143893 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919e86bf-77a2-4f2a-ad72-2470a973df67 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:37.152313 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-bb8c65eb-b4ea-416b-8159-35abe098b4e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03fcad2f720446899501463f77edb9f1'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:37.237418 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-bb8c65eb-b4ea-416b-8159-35abe098b4e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03fcad2f720446899501463f77edb9f1'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:37.251144 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1210/2428] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 09:05:36 2022] DELETE /identity/v3/users/03fcad2f720446899501463f77edb9f1 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:05:37.255847 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9db54a-5af6-44bd-a98a-743e092206d1 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:37.258440 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9db54a-5af6-44bd-a98a-743e092206d1 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:37.258440 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9db54a-5af6-44bd-a98a-743e092206d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:37.276983 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-8a9db54a-5af6-44bd-a98a-743e092206d1 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:05:37.278117 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-8a9db54a-5af6-44bd-a98a-743e092206d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:37.340715 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1211/2429] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:05:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:05:37.351122 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9db54a-5af6-44bd-a98a-743e092206d1 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:37.352626 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-8a9db54a-5af6-44bd-a98a-743e092206d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:37.385465 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-919e86bf-77a2-4f2a-ad72-2470a973df67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6eb4015e1c38446daeeeb884e5495870'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:37.389786 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-919e86bf-77a2-4f2a-ad72-2470a973df67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6eb4015e1c38446daeeeb884e5495870'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:37.389786 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-919e86bf-77a2-4f2a-ad72-2470a973df67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6eb4015e1c38446daeeeb884e5495870'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:37.400531 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-919e86bf-77a2-4f2a-ad72-2470a973df67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6eb4015e1c38446daeeeb884e5495870'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:37.409522 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:05:37.415079 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9db54a-5af6-44bd-a98a-743e092206d1 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:37.418900 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c91428c-0cd8-4493-aa5a-1cbbc4d04907 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:37.419520 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c91428c-0cd8-4493-aa5a-1cbbc4d04907 service neutron] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:37.419804 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c91428c-0cd8-4493-aa5a-1cbbc4d04907 service neutron] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:37.420092 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c91428c-0cd8-4493-aa5a-1cbbc4d04907 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:37.421910 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-4c91428c-0cd8-4493-aa5a-1cbbc4d04907 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:37.435427 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c91428c-0cd8-4493-aa5a-1cbbc4d04907 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:37.437111 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c91428c-0cd8-4493-aa5a-1cbbc4d04907 service neutron] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:37.448522 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1212/2430] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:05:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 09:05:37.483931 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-919e86bf-77a2-4f2a-ad72-2470a973df67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6eb4015e1c38446daeeeb884e5495870'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:37.504921 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1219/2431] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 09:05:37 2022] DELETE /identity/v3/users/6eb4015e1c38446daeeeb884e5495870 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:05:37.821169 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c91428c-0cd8-4493-aa5a-1cbbc4d04907 service neutron] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:37.822911 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-4c91428c-0cd8-4493-aa5a-1cbbc4d04907 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:37.840378 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c91428c-0cd8-4493-aa5a-1cbbc4d04907 service neutron] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:37.921227 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cb9759-08a3-445d-bdac-9af55c51bd1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:37.921857 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cb9759-08a3-445d-bdac-9af55c51bd1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:37.922141 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cb9759-08a3-445d-bdac-9af55c51bd1c admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:37.922411 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cb9759-08a3-445d-bdac-9af55c51bd1c admin admin] PATH_INFO: `/v3/projects/c2ee9cd323d14e18a73eaf58f8c71870` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:37.923580 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cb9759-08a3-445d-bdac-9af55c51bd1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2ee9cd323d14e18a73eaf58f8c71870)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:37.925555 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cb9759-08a3-445d-bdac-9af55c51bd1c admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:38.005579 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919e86bf-77a2-4f2a-ad72-2470a973df67 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:38.007118 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-919e86bf-77a2-4f2a-ad72-2470a973df67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:38.021420 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919e86bf-77a2-4f2a-ad72-2470a973df67 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:38.026493 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c025fdc3-fbe7-4813-9157-bea858c86cb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:38.027153 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c025fdc3-fbe7-4813-9157-bea858c86cb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:38.027431 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c025fdc3-fbe7-4813-9157-bea858c86cb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:38.027707 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c025fdc3-fbe7-4813-9157-bea858c86cb4 admin admin] PATH_INFO: `/v3/projects/84efd7e61a0b4ee8a3c8394af63c3ff0` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:38.028900 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c025fdc3-fbe7-4813-9157-bea858c86cb4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=84efd7e61a0b4ee8a3c8394af63c3ff0)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:38.034428 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c025fdc3-fbe7-4813-9157-bea858c86cb4 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:38.059587 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-b8cb9759-08a3-445d-bdac-9af55c51bd1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2ee9cd323d14e18a73eaf58f8c71870', 'request_id': 'req-b8cb9759-08a3-445d-bdac-9af55c51bd1c'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:38.163883 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-b8cb9759-08a3-445d-bdac-9af55c51bd1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2ee9cd323d14e18a73eaf58f8c71870', 'request_id': 'req-b8cb9759-08a3-445d-bdac-9af55c51bd1c'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:38.170408 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:05:38.174452 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-b8cb9759-08a3-445d-bdac-9af55c51bd1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2ee9cd323d14e18a73eaf58f8c71870', 'request_id': 'req-b8cb9759-08a3-445d-bdac-9af55c51bd1c'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:38.190849 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-b8cb9759-08a3-445d-bdac-9af55c51bd1c admin admin] The token cache is being invalidate because project c2ee9cd323d14e18a73eaf58f8c71870 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:05:38.191167 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-b8cb9759-08a3-445d-bdac-9af55c51bd1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:38.192368 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-b8cb9759-08a3-445d-bdac-9af55c51bd1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:38.192493 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-c025fdc3-fbe7-4813-9157-bea858c86cb4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84efd7e61a0b4ee8a3c8394af63c3ff0', 'request_id': 'req-c025fdc3-fbe7-4813-9157-bea858c86cb4'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:38.195221 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1213/2432] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 09:05:37 2022] DELETE /identity/v3/projects/c2ee9cd323d14e18a73eaf58f8c71870 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:05:38.199600 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-c025fdc3-fbe7-4813-9157-bea858c86cb4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84efd7e61a0b4ee8a3c8394af63c3ff0', 'request_id': 'req-c025fdc3-fbe7-4813-9157-bea858c86cb4'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:38.200081 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdce4c7-bbb9-4819-86b4-2a4bcb1b0dd1 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:38.200367 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdce4c7-bbb9-4819-86b4-2a4bcb1b0dd1 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:38.200707 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdce4c7-bbb9-4819-86b4-2a4bcb1b0dd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:38.222928 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-6fdce4c7-bbb9-4819-86b4-2a4bcb1b0dd1 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:05:38.225053 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-6fdce4c7-bbb9-4819-86b4-2a4bcb1b0dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:38.291417 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1214/2433] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:05:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:05:38.299664 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9962c94d-db5c-44d2-b7e9-2bc007c6a40d None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:38.299945 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9962c94d-db5c-44d2-b7e9-2bc007c6a40d None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:38.300219 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9962c94d-db5c-44d2-b7e9-2bc007c6a40d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:38.318314 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-c025fdc3-fbe7-4813-9157-bea858c86cb4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84efd7e61a0b4ee8a3c8394af63c3ff0', 'request_id': 'req-c025fdc3-fbe7-4813-9157-bea858c86cb4'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:38.322291 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-9962c94d-db5c-44d2-b7e9-2bc007c6a40d None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:05:38.323602 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-9962c94d-db5c-44d2-b7e9-2bc007c6a40d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:38.331056 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-c025fdc3-fbe7-4813-9157-bea858c86cb4 admin admin] The token cache is being invalidate because project 84efd7e61a0b4ee8a3c8394af63c3ff0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:05:38.331634 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-c025fdc3-fbe7-4813-9157-bea858c86cb4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:38.332419 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-c025fdc3-fbe7-4813-9157-bea858c86cb4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:05:38.333231 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1215/2434] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:05:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:05:38.333903 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1220/2435] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 09:05:38 2022] DELETE /identity/v3/projects/84efd7e61a0b4ee8a3c8394af63c3ff0 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:05:38.352810 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9962c94d-db5c-44d2-b7e9-2bc007c6a40d None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:38.352810 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9962c94d-db5c-44d2-b7e9-2bc007c6a40d None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:38.358522 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613e4489-d1cd-4945-8d0a-aaa259d97ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:38.359548 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613e4489-d1cd-4945-8d0a-aaa259d97ed6 admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:38.360057 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613e4489-d1cd-4945-8d0a-aaa259d97ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:38.370414 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613e4489-d1cd-4945-8d0a-aaa259d97ed6 admin admin] PATH_INFO: `/v3/domains` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:38.370414 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613e4489-d1cd-4945-8d0a-aaa259d97ed6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:05:38.370414 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613e4489-d1cd-4945-8d0a-aaa259d97ed6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:38.370414 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613e4489-d1cd-4945-8d0a-aaa259d97ed6 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:38.375999 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:05:38.386876 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1216/2436] 10.222.0.55 () {58 vars in 1153 bytes} [Wed Aug 24 09:05:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:05:38.391452 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c025fdc3-fbe7-4813-9157-bea858c86cb4 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:38.391452 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-c025fdc3-fbe7-4813-9157-bea858c86cb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:38.413363 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c025fdc3-fbe7-4813-9157-bea858c86cb4 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:38.433489 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8bbebb-c14f-423a-9051-65c6a9c8e3ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:38.434775 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8bbebb-c14f-423a-9051-65c6a9c8e3ac admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:38.435526 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8bbebb-c14f-423a-9051-65c6a9c8e3ac admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:38.436268 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8bbebb-c14f-423a-9051-65c6a9c8e3ac admin admin] PATH_INFO: `/v3/projects` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:38.437444 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8bbebb-c14f-423a-9051-65c6a9c8e3ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:38.440725 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8bbebb-c14f-423a-9051-65c6a9c8e3ac admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:38.641631 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1221/2437] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 09:05:38 2022] POST /identity/v3/projects => generated 371 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:05:38.649167 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613e4489-d1cd-4945-8d0a-aaa259d97ed6 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:38.663557 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613e4489-d1cd-4945-8d0a-aaa259d97ed6 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:38.730936 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef02bd3-41de-48a3-989e-a93e2bbd2fe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:38.731567 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef02bd3-41de-48a3-989e-a93e2bbd2fe4 admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:38.731845 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef02bd3-41de-48a3-989e-a93e2bbd2fe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:38.732118 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef02bd3-41de-48a3-989e-a93e2bbd2fe4 admin admin] PATH_INFO: `/v3/users` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:38.732813 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef02bd3-41de-48a3-989e-a93e2bbd2fe4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:38.734839 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef02bd3-41de-48a3-989e-a93e2bbd2fe4 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:38.843991 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1217/2438] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 09:05:38 2022] POST /identity/v3/users => generated 286 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:05:38.848864 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8bbebb-c14f-423a-9051-65c6a9c8e3ac admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:38.860147 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8bbebb-c14f-423a-9051-65c6a9c8e3ac admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:38.864837 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d1389b-22c2-4a50-abf3-828ae84acfdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:38.865446 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d1389b-22c2-4a50-abf3-828ae84acfdb admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:38.865721 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d1389b-22c2-4a50-abf3-828ae84acfdb admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:38.865989 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d1389b-22c2-4a50-abf3-828ae84acfdb admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:38.866635 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d1389b-22c2-4a50-abf3-828ae84acfdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:38.868811 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d1389b-22c2-4a50-abf3-828ae84acfdb admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:38.879714 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1222/2439] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:05:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:05:38.884482 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef02bd3-41de-48a3-989e-a93e2bbd2fe4 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:38.895351 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef02bd3-41de-48a3-989e-a93e2bbd2fe4 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:38.900726 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2900a7f4-ff2b-4675-9b82-9fd1235467e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:38.901424 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2900a7f4-ff2b-4675-9b82-9fd1235467e7 admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:38.901706 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2900a7f4-ff2b-4675-9b82-9fd1235467e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:38.901970 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2900a7f4-ff2b-4675-9b82-9fd1235467e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:38.902615 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2900a7f4-ff2b-4675-9b82-9fd1235467e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:38.904223 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2900a7f4-ff2b-4675-9b82-9fd1235467e7 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:38.916834 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1218/2440] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:05:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:05:38.921085 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d1389b-22c2-4a50-abf3-828ae84acfdb admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:38.931733 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d1389b-22c2-4a50-abf3-828ae84acfdb admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:38.937279 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61351420-50e0-4b9a-8eec-ebceafa650a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:38.937943 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61351420-50e0-4b9a-8eec-ebceafa650a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:38.938229 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61351420-50e0-4b9a-8eec-ebceafa650a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:38.938501 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61351420-50e0-4b9a-8eec-ebceafa650a2 admin admin] PATH_INFO: `/v3/projects/e19c750fc9af412f93ff6369e4c57730/users/faf0d3eebf714e8891c6a3eebf8d87a9/roles/b674fd0a592e4e0cada811bba5ceea4d` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:38.954632 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61351420-50e0-4b9a-8eec-ebceafa650a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e19c750fc9af412f93ff6369e4c57730, user_id=faf0d3eebf714e8891c6a3eebf8d87a9, role_id=b674fd0a592e4e0cada811bba5ceea4d)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:38.957403 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61351420-50e0-4b9a-8eec-ebceafa650a2 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:39.072347 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1223/2441] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 09:05:38 2022] PUT /identity/v3/projects/e19c750fc9af412f93ff6369e4c57730/users/faf0d3eebf714e8891c6a3eebf8d87a9/roles/b674fd0a592e4e0cada811bba5ceea4d => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:05:39.077567 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8a5c89-4819-44b3-99a7-2e47eec98f4d None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:39.077850 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8a5c89-4819-44b3-99a7-2e47eec98f4d None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:39.078120 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8a5c89-4819-44b3-99a7-2e47eec98f4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:39.098989 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-af8a5c89-4819-44b3-99a7-2e47eec98f4d None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:05:39.100216 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-af8a5c89-4819-44b3-99a7-2e47eec98f4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:39.167695 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1219/2442] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:05:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:05:39.178004 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61351420-50e0-4b9a-8eec-ebceafa650a2 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:39.179794 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-61351420-50e0-4b9a-8eec-ebceafa650a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:39.249526 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61351420-50e0-4b9a-8eec-ebceafa650a2 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:39.253531 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a833c4e6-aff5-4b50-8bfa-86502250a245 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:39.254139 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a833c4e6-aff5-4b50-8bfa-86502250a245 service neutron] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:39.254413 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a833c4e6-aff5-4b50-8bfa-86502250a245 service neutron] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:39.254684 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a833c4e6-aff5-4b50-8bfa-86502250a245 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:39.265049 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a833c4e6-aff5-4b50-8bfa-86502250a245 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:39.266497 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a833c4e6-aff5-4b50-8bfa-86502250a245 service neutron] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:39.277273 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1224/2443] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:05:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:05:40.702047 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8a5c89-4819-44b3-99a7-2e47eec98f4d None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:40.703625 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-af8a5c89-4819-44b3-99a7-2e47eec98f4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:40.788426 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8a5c89-4819-44b3-99a7-2e47eec98f4d None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:40.795081 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97898220-ea44-4bd5-bac1-229a2e4dfb3e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbb9ac667f04c35a63b2689df18efed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:40.796258 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97898220-ea44-4bd5-bac1-229a2e4dfb3e service cinder] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:40.796812 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97898220-ea44-4bd5-bac1-229a2e4dfb3e service cinder] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:40.797912 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97898220-ea44-4bd5-bac1-229a2e4dfb3e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:40.800498 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-97898220-ea44-4bd5-bac1-229a2e4dfb3e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:40.870459 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:05:40.915391 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97898220-ea44-4bd5-bac1-229a2e4dfb3e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:40.916800 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97898220-ea44-4bd5-bac1-229a2e4dfb3e service cinder] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:40.928881 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1220/2444] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:05:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:05:45.607204 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512bc078-99fd-4e86-850f-8dedb3c1398d None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:45.607756 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512bc078-99fd-4e86-850f-8dedb3c1398d None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:45.607756 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512bc078-99fd-4e86-850f-8dedb3c1398d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:45.639586 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-512bc078-99fd-4e86-850f-8dedb3c1398d None None] MFA Rules not processed for user `faf0d3eebf714e8891c6a3eebf8d87a9`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:05:45.640932 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-512bc078-99fd-4e86-850f-8dedb3c1398d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:45.708722 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:05:45.726948 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1225/2445] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:05:45 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:05:45.731855 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97898220-ea44-4bd5-bac1-229a2e4dfb3e service cinder] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:45.743562 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97898220-ea44-4bd5-bac1-229a2e4dfb3e service cinder] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:45.748916 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce4e304-2f9b-4214-8151-5ec0ff7a3d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:45.749746 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce4e304-2f9b-4214-8151-5ec0ff7a3d53 admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:45.750114 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce4e304-2f9b-4214-8151-5ec0ff7a3d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:45.750523 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce4e304-2f9b-4214-8151-5ec0ff7a3d53 admin admin] PATH_INFO: `/v3/projects` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:45.751470 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce4e304-2f9b-4214-8151-5ec0ff7a3d53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:45.753970 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce4e304-2f9b-4214-8151-5ec0ff7a3d53 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:45.885761 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1221/2446] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 09:05:45 2022] POST /identity/v3/projects => generated 373 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:05:45.889762 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512bc078-99fd-4e86-850f-8dedb3c1398d None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:45.901765 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512bc078-99fd-4e86-850f-8dedb3c1398d None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:45.963758 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9000b8c-e0e2-4fb0-af67-c97958cad19a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:45.964285 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9000b8c-e0e2-4fb0-af67-c97958cad19a admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:45.964588 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9000b8c-e0e2-4fb0-af67-c97958cad19a admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:45.964863 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9000b8c-e0e2-4fb0-af67-c97958cad19a admin admin] PATH_INFO: `/v3/users` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:45.965504 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9000b8c-e0e2-4fb0-af67-c97958cad19a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:45.967426 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9000b8c-e0e2-4fb0-af67-c97958cad19a admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:46.090955 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1226/2447] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 09:05:45 2022] POST /identity/v3/users => generated 293 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:05:46.095545 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce4e304-2f9b-4214-8151-5ec0ff7a3d53 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:46.107956 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce4e304-2f9b-4214-8151-5ec0ff7a3d53 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:46.112869 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a2f552-ad2d-45d2-9d1b-796be8ab31ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:46.113355 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a2f552-ad2d-45d2-9d1b-796be8ab31ae admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:46.113634 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a2f552-ad2d-45d2-9d1b-796be8ab31ae admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:46.113974 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a2f552-ad2d-45d2-9d1b-796be8ab31ae admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:46.114556 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a2f552-ad2d-45d2-9d1b-796be8ab31ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:46.116211 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a2f552-ad2d-45d2-9d1b-796be8ab31ae admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:46.129307 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1222/2448] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:05:46 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:05:46.133455 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9000b8c-e0e2-4fb0-af67-c97958cad19a admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:46.145139 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9000b8c-e0e2-4fb0-af67-c97958cad19a admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:46.150337 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942ac0c3-c865-4d5b-aa26-158675ccabe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:46.151029 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942ac0c3-c865-4d5b-aa26-158675ccabe5 admin admin] REQUEST_METHOD: `PUT` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:46.151307 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942ac0c3-c865-4d5b-aa26-158675ccabe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:46.151587 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942ac0c3-c865-4d5b-aa26-158675ccabe5 admin admin] PATH_INFO: `/v3/projects/d8ad005a17d043ecbbab5cf0ecc11ac8/users/0afee2bcfef34b6bb64d0bfe4dc7dab9/roles/b870cd1acaa744e1ae1d58b9ac283821` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:46.172684 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942ac0c3-c865-4d5b-aa26-158675ccabe5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8ad005a17d043ecbbab5cf0ecc11ac8, user_id=0afee2bcfef34b6bb64d0bfe4dc7dab9, role_id=b870cd1acaa744e1ae1d58b9ac283821)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:46.177089 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942ac0c3-c865-4d5b-aa26-158675ccabe5 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:46.261526 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1227/2449] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 09:05:46 2022] PUT /identity/v3/projects/d8ad005a17d043ecbbab5cf0ecc11ac8/users/0afee2bcfef34b6bb64d0bfe4dc7dab9/roles/b870cd1acaa744e1ae1d58b9ac283821 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:05:46.265912 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a2f552-ad2d-45d2-9d1b-796be8ab31ae admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:46.291855 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a2f552-ad2d-45d2-9d1b-796be8ab31ae admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:46.334639 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942ac0c3-c865-4d5b-aa26-158675ccabe5 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:46.337903 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-942ac0c3-c865-4d5b-aa26-158675ccabe5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:46.399383 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49c7b01-f7a7-43a4-b6bb-3641a6983fb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:46.400642 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49c7b01-f7a7-43a4-b6bb-3641a6983fb4 admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:46.401016 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49c7b01-f7a7-43a4-b6bb-3641a6983fb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:46.401391 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49c7b01-f7a7-43a4-b6bb-3641a6983fb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:46.403477 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49c7b01-f7a7-43a4-b6bb-3641a6983fb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:46.405032 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49c7b01-f7a7-43a4-b6bb-3641a6983fb4 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:46.419746 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942ac0c3-c865-4d5b-aa26-158675ccabe5 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:46.420616 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1223/2450] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:05:46 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:05:46.425595 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49c7b01-f7a7-43a4-b6bb-3641a6983fb4 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:46.432299 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c268db47-aeda-440c-b438-759467c16a95 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbb9ac667f04c35a63b2689df18efed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:46.432299 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c268db47-aeda-440c-b438-759467c16a95 service cinder] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:46.432299 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c268db47-aeda-440c-b438-759467c16a95 service cinder] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:46.432299 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c268db47-aeda-440c-b438-759467c16a95 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:46.433660 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-c268db47-aeda-440c-b438-759467c16a95 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:46.438474 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49c7b01-f7a7-43a4-b6bb-3641a6983fb4 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:46.443714 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3de01e-db43-4d75-a61d-fe3cf69d4daa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:46.444561 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3de01e-db43-4d75-a61d-fe3cf69d4daa admin admin] REQUEST_METHOD: `PUT` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:46.445024 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3de01e-db43-4d75-a61d-fe3cf69d4daa admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:46.445393 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3de01e-db43-4d75-a61d-fe3cf69d4daa admin admin] PATH_INFO: `/v3/projects/d8ad005a17d043ecbbab5cf0ecc11ac8/users/0afee2bcfef34b6bb64d0bfe4dc7dab9/roles/b870cd1acaa744e1ae1d58b9ac283821` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:46.447641 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3de01e-db43-4d75-a61d-fe3cf69d4daa admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8ad005a17d043ecbbab5cf0ecc11ac8, user_id=0afee2bcfef34b6bb64d0bfe4dc7dab9, role_id=b870cd1acaa744e1ae1d58b9ac283821)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:46.451100 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3de01e-db43-4d75-a61d-fe3cf69d4daa admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:46.467901 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1224/2451] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 09:05:46 2022] PUT /identity/v3/projects/d8ad005a17d043ecbbab5cf0ecc11ac8/users/0afee2bcfef34b6bb64d0bfe4dc7dab9/roles/b870cd1acaa744e1ae1d58b9ac283821 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:05:46.576200 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:05:46.580822 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c268db47-aeda-440c-b438-759467c16a95 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:46.582286 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c268db47-aeda-440c-b438-759467c16a95 service cinder] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:46.606776 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1228/2452] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:05:46 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 09:05:51.233057 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3de01e-db43-4d75-a61d-fe3cf69d4daa admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:51.360327 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3de01e-db43-4d75-a61d-fe3cf69d4daa admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:51.373387 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c268db47-aeda-440c-b438-759467c16a95 service cinder] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:51.373971 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e292a6-7db9-4dfb-b074-1089280f084b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbb9ac667f04c35a63b2689df18efed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:51.374837 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e292a6-7db9-4dfb-b074-1089280f084b service cinder] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:51.375203 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e292a6-7db9-4dfb-b074-1089280f084b service cinder] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:51.375576 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e292a6-7db9-4dfb-b074-1089280f084b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:51.376230 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-c268db47-aeda-440c-b438-759467c16a95 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:51.384707 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-95e292a6-7db9-4dfb-b074-1089280f084b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:51.464871 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:05:51.551447 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c268db47-aeda-440c-b438-759467c16a95 service cinder] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:51.572455 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e292a6-7db9-4dfb-b074-1089280f084b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:51.575456 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e292a6-7db9-4dfb-b074-1089280f084b service cinder] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:51.617542 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1225/2453] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:05:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:05:51.683110 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5ed655-852a-4f30-9476-f0db30a0daa0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17da357e555420799ee681bffbcc268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:51.683745 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5ed655-852a-4f30-9476-f0db30a0daa0 service nova] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:51.684593 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5ed655-852a-4f30-9476-f0db30a0daa0 service nova] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:51.684593 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5ed655-852a-4f30-9476-f0db30a0daa0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:51.688669 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-0a5ed655-852a-4f30-9476-f0db30a0daa0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:51.804291 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5ed655-852a-4f30-9476-f0db30a0daa0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:51.806182 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5ed655-852a-4f30-9476-f0db30a0daa0 service nova] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:51.812232 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:05:51.840010 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1229/2454] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:05:51 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:05:53.453836 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebfe759-8730-43a5-b926-c5307d8f2c17 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:53.454308 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebfe759-8730-43a5-b926-c5307d8f2c17 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:53.454517 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebfe759-8730-43a5-b926-c5307d8f2c17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:53.501490 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-cebfe759-8730-43a5-b926-c5307d8f2c17 None None] MFA Rules not processed for user `0afee2bcfef34b6bb64d0bfe4dc7dab9`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:05:53.505119 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-cebfe759-8730-43a5-b926-c5307d8f2c17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:53.611253 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1226/2455] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:05:53 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:05:53.626566 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5ed655-852a-4f30-9476-f0db30a0daa0 service nova] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:53.643350 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5ed655-852a-4f30-9476-f0db30a0daa0 service nova] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:53.650694 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9504c1e-cf5e-496f-aaa1-4b03cd1edddd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17da357e555420799ee681bffbcc268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:53.651504 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9504c1e-cf5e-496f-aaa1-4b03cd1edddd service nova] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:53.651793 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9504c1e-cf5e-496f-aaa1-4b03cd1edddd service nova] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:53.652073 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9504c1e-cf5e-496f-aaa1-4b03cd1edddd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:53.767271 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9504c1e-cf5e-496f-aaa1-4b03cd1edddd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:53.768997 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9504c1e-cf5e-496f-aaa1-4b03cd1edddd service nova] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:53.780250 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1230/2456] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:05:53 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:05:57.167647 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabc8f46-2e86-4075-879b-0b47c448d15c None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:57.168203 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabc8f46-2e86-4075-879b-0b47c448d15c None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:57.168203 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabc8f46-2e86-4075-879b-0b47c448d15c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:57.188314 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-fabc8f46-2e86-4075-879b-0b47c448d15c None None] MFA Rules not processed for user `faf0d3eebf714e8891c6a3eebf8d87a9`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:05:57.190226 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-fabc8f46-2e86-4075-879b-0b47c448d15c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:57.205702 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1227/2457] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:05:57 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:06:03.775578 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=32 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:06:05.639918 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921283d6-663e-491e-899d-6f57c3771751 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:06:05.640627 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921283d6-663e-491e-899d-6f57c3771751 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:06:05.640627 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921283d6-663e-491e-899d-6f57c3771751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:06:05.670702 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-921283d6-663e-491e-899d-6f57c3771751 None None] MFA Rules not processed for user `993697d33b7841668b1eabcdcd6b40f9`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:06:05.671852 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-921283d6-663e-491e-899d-6f57c3771751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:06:05.732174 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1231/2458] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:06:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:06:07.182362 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=59 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:06:09.760776 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabc8f46-2e86-4075-879b-0b47c448d15c None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:06:09.762355 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-fabc8f46-2e86-4075-879b-0b47c448d15c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:06:09.823253 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabc8f46-2e86-4075-879b-0b47c448d15c None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:06:09.827211 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36ad948-aa8e-403e-bccc-03d8035d4cb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:06:09.827822 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36ad948-aa8e-403e-bccc-03d8035d4cb2 service neutron] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:06:09.828102 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36ad948-aa8e-403e-bccc-03d8035d4cb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:06:09.828413 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36ad948-aa8e-403e-bccc-03d8035d4cb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:06:09.830134 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-d36ad948-aa8e-403e-bccc-03d8035d4cb2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:06:09.841104 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36ad948-aa8e-403e-bccc-03d8035d4cb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:06:09.842546 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36ad948-aa8e-403e-bccc-03d8035d4cb2 service neutron] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:06:09.851808 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1228/2459] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:06:09 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:06:15.725070 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=28 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:06:19.847487 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=26 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:06:19.889443 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921283d6-663e-491e-899d-6f57c3771751 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:06:19.891216 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-921283d6-663e-491e-899d-6f57c3771751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:06:19.915894 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921283d6-663e-491e-899d-6f57c3771751 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:06:19.922490 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47174163-8d4e-4e44-bd45-63dffba81241 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17da357e555420799ee681bffbcc268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:06:19.923250 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47174163-8d4e-4e44-bd45-63dffba81241 service nova] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:06:19.923561 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47174163-8d4e-4e44-bd45-63dffba81241 service nova] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:06:19.923903 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47174163-8d4e-4e44-bd45-63dffba81241 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:06:19.926360 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-47174163-8d4e-4e44-bd45-63dffba81241 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:06:20.050081 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47174163-8d4e-4e44-bd45-63dffba81241 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:06:20.051776 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47174163-8d4e-4e44-bd45-63dffba81241 service nova] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:06:20.071473 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1232/2460] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:06:19 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:06:25.920579 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36ad948-aa8e-403e-bccc-03d8035d4cb2 service neutron] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:06:25.935402 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36ad948-aa8e-403e-bccc-03d8035d4cb2 service neutron] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:06:25.948080 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e948df5-5bef-41d2-b4fb-0544df096ce2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17da357e555420799ee681bffbcc268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:06:25.948868 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e948df5-5bef-41d2-b4fb-0544df096ce2 service nova] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:06:25.949227 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e948df5-5bef-41d2-b4fb-0544df096ce2 service nova] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:06:25.949573 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e948df5-5bef-41d2-b4fb-0544df096ce2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:06:25.951703 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-5e948df5-5bef-41d2-b4fb-0544df096ce2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:06:25.966904 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e948df5-5bef-41d2-b4fb-0544df096ce2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:06:25.968787 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e948df5-5bef-41d2-b4fb-0544df096ce2 service nova] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:06:25.984747 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1229/2461] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:06:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:06:26.461530 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47174163-8d4e-4e44-bd45-63dffba81241 service nova] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:06:26.462259 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-47174163-8d4e-4e44-bd45-63dffba81241 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:06:26.551126 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47174163-8d4e-4e44-bd45-63dffba81241 service nova] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:06:26.557148 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96497064-daa7-4403-9c8f-d8aef8447347 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf0c99771661413aaa0f237af6b394ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:06:26.557477 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96497064-daa7-4403-9c8f-d8aef8447347 service placement] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:06:26.557757 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96497064-daa7-4403-9c8f-d8aef8447347 service placement] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:06:26.558024 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96497064-daa7-4403-9c8f-d8aef8447347 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:06:26.559834 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-96497064-daa7-4403-9c8f-d8aef8447347 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:06:26.572106 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96497064-daa7-4403-9c8f-d8aef8447347 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:06:26.575051 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96497064-daa7-4403-9c8f-d8aef8447347 service placement] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:06:26.585524 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1233/2462] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:06:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:06:27.828394 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e05a70f-344a-40ad-a5cb-f0ce525f6f6b None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:06:27.828917 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e05a70f-344a-40ad-a5cb-f0ce525f6f6b None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:06:27.829774 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e05a70f-344a-40ad-a5cb-f0ce525f6f6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:06:27.887971 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-8e05a70f-344a-40ad-a5cb-f0ce525f6f6b None None] MFA Rules not processed for user `f652a368b1cc42ae8b6f6c7a6dff6f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:06:27.889231 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-8e05a70f-344a-40ad-a5cb-f0ce525f6f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:06:27.955296 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1230/2463] 10.222.0.55 () {58 vars in 1009 bytes} [Wed Aug 24 09:06:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:06:28.082759 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96497064-daa7-4403-9c8f-d8aef8447347 service placement] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:06:28.083414 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-96497064-daa7-4403-9c8f-d8aef8447347 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:06:28.162331 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:06:28.200197 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96497064-daa7-4403-9c8f-d8aef8447347 service placement] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:06:28.206285 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6185b962-7cd1-4b8b-b44a-ca0b33c33f4c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0ebb5ccabef45378f0c1da69721e051', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:06:28.206916 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6185b962-7cd1-4b8b-b44a-ca0b33c33f4c service swift] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:06:28.207200 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6185b962-7cd1-4b8b-b44a-ca0b33c33f4c service swift] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:06:28.207476 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6185b962-7cd1-4b8b-b44a-ca0b33c33f4c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:06:28.216863 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6185b962-7cd1-4b8b-b44a-ca0b33c33f4c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:06:28.218540 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6185b962-7cd1-4b8b-b44a-ca0b33c33f4c service swift] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:06:28.232808 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1234/2464] 10.222.0.55 () {60 vars in 1384 bytes} [Wed Aug 24 09:06:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 09:06:37.952042 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=46 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:06:38.244294 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=14 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:06:49.400851 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e05a70f-344a-40ad-a5cb-f0ce525f6f6b None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:06:49.410489 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e05a70f-344a-40ad-a5cb-f0ce525f6f6b None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:06:49.414228 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115134ce-512a-4a4c-b641-ef36b953c65f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:06:49.414828 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115134ce-512a-4a4c-b641-ef36b953c65f service neutron] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:06:49.415105 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115134ce-512a-4a4c-b641-ef36b953c65f service neutron] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:06:49.415377 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115134ce-512a-4a4c-b641-ef36b953c65f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:06:49.425217 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115134ce-512a-4a4c-b641-ef36b953c65f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:06:49.426845 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115134ce-512a-4a4c-b641-ef36b953c65f service neutron] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:06:49.438879 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1231/2465] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:06:49 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:06:50.138049 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6185b962-7cd1-4b8b-b44a-ca0b33c33f4c service swift] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:06:50.140715 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-6185b962-7cd1-4b8b-b44a-ca0b33c33f4c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:06:50.225687 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6185b962-7cd1-4b8b-b44a-ca0b33c33f4c service swift] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:06:50.237815 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36da64f4-35d6-4dc4-b9ed-11781fdc5f7a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5512043ca9a7421b821741ca88d50b07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:06:50.238917 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36da64f4-35d6-4dc4-b9ed-11781fdc5f7a service glance] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:06:50.239377 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36da64f4-35d6-4dc4-b9ed-11781fdc5f7a service glance] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:06:50.239818 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36da64f4-35d6-4dc4-b9ed-11781fdc5f7a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:06:50.247776 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-36da64f4-35d6-4dc4-b9ed-11781fdc5f7a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:06:50.258548 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36da64f4-35d6-4dc4-b9ed-11781fdc5f7a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:06:50.260752 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36da64f4-35d6-4dc4-b9ed-11781fdc5f7a service glance] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:06:50.287315 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1235/2466] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:06:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:06:50.540177 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115134ce-512a-4a4c-b641-ef36b953c65f service neutron] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:06:50.552336 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115134ce-512a-4a4c-b641-ef36b953c65f service neutron] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:06:50.558000 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7673657-42b4-421e-80e8-7705d51cef41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:06:50.558920 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7673657-42b4-421e-80e8-7705d51cef41 service neutron] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:06:50.559503 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7673657-42b4-421e-80e8-7705d51cef41 service neutron] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:06:50.560008 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7673657-42b4-421e-80e8-7705d51cef41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:06:50.562433 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-b7673657-42b4-421e-80e8-7705d51cef41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:06:50.575331 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7673657-42b4-421e-80e8-7705d51cef41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:06:50.577040 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7673657-42b4-421e-80e8-7705d51cef41 service neutron] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:06:50.586852 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1232/2467] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:06:50 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:06:50.664222 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36da64f4-35d6-4dc4-b9ed-11781fdc5f7a service glance] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:06:50.676446 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36da64f4-35d6-4dc4-b9ed-11781fdc5f7a service glance] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:06:50.681106 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646b1d27-b4aa-4b4e-82d0-96ab3d5d3d4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:06:50.682422 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646b1d27-b4aa-4b4e-82d0-96ab3d5d3d4b service neutron] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:06:50.682638 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646b1d27-b4aa-4b4e-82d0-96ab3d5d3d4b service neutron] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:06:50.682904 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646b1d27-b4aa-4b4e-82d0-96ab3d5d3d4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:06:50.684745 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-646b1d27-b4aa-4b4e-82d0-96ab3d5d3d4b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:06:50.714768 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646b1d27-b4aa-4b4e-82d0-96ab3d5d3d4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:06:50.717245 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646b1d27-b4aa-4b4e-82d0-96ab3d5d3d4b service neutron] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:06:50.731153 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1236/2468] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:06:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:06:51.923457 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7673657-42b4-421e-80e8-7705d51cef41 service neutron] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:06:51.938387 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7673657-42b4-421e-80e8-7705d51cef41 service neutron] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:06:51.944123 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f57adf-5a9f-430d-b969-5ab6660ee546 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17da357e555420799ee681bffbcc268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:06:51.944787 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f57adf-5a9f-430d-b969-5ab6660ee546 service nova] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:06:51.945061 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f57adf-5a9f-430d-b969-5ab6660ee546 service nova] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:06:51.945324 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f57adf-5a9f-430d-b969-5ab6660ee546 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:06:51.947510 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-22f57adf-5a9f-430d-b969-5ab6660ee546 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:06:51.964597 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f57adf-5a9f-430d-b969-5ab6660ee546 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:06:51.967015 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f57adf-5a9f-430d-b969-5ab6660ee546 service nova] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:06:51.979504 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1233/2469] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:06:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:06:57.010169 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a0e124-4dcc-422f-b640-fca6dfe7d22f None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:06:57.010652 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a0e124-4dcc-422f-b640-fca6dfe7d22f None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:06:57.010888 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a0e124-4dcc-422f-b640-fca6dfe7d22f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:06:57.042146 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-40a0e124-4dcc-422f-b640-fca6dfe7d22f None None] MFA Rules not processed for user `f652a368b1cc42ae8b6f6c7a6dff6f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:06:57.043950 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-40a0e124-4dcc-422f-b640-fca6dfe7d22f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:06:57.052219 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1237/2470] 10.222.0.55 () {58 vars in 1009 bytes} [Wed Aug 24 09:06:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:06:57.066859 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f57adf-5a9f-430d-b969-5ab6660ee546 service nova] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:06:57.078495 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f57adf-5a9f-430d-b969-5ab6660ee546 service nova] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:06:57.085282 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c0e53d-e4f0-4204-8192-e03f6641c377 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0ebb5ccabef45378f0c1da69721e051', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:06:57.086016 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c0e53d-e4f0-4204-8192-e03f6641c377 service swift] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:06:57.086302 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c0e53d-e4f0-4204-8192-e03f6641c377 service swift] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:06:57.086740 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c0e53d-e4f0-4204-8192-e03f6641c377 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:06:57.097310 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c0e53d-e4f0-4204-8192-e03f6641c377 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:06:57.099726 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c0e53d-e4f0-4204-8192-e03f6641c377 service swift] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:06:57.111692 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1234/2471] 10.222.0.55 () {60 vars in 1384 bytes} [Wed Aug 24 09:06:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 09:07:03.270712 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bd9d0e-f88f-4608-b38e-191df23afc09 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:03.271359 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bd9d0e-f88f-4608-b38e-191df23afc09 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:03.271359 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bd9d0e-f88f-4608-b38e-191df23afc09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:03.292219 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-b1bd9d0e-f88f-4608-b38e-191df23afc09 None None] MFA Rules not processed for user `c2c3752027f342a489e6eb9d3b540819`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:03.294081 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-b1bd9d0e-f88f-4608-b38e-191df23afc09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:03.356913 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1238/2472] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:07:03 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:07:03.704558 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c0e53d-e4f0-4204-8192-e03f6641c377 service swift] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:03.714960 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c0e53d-e4f0-4204-8192-e03f6641c377 service swift] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:03.720734 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbd2fc3-6f1c-414d-82a4-c9f79e962811 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:03.721380 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbd2fc3-6f1c-414d-82a4-c9f79e962811 service neutron] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:03.721716 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbd2fc3-6f1c-414d-82a4-c9f79e962811 service neutron] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:03.721955 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbd2fc3-6f1c-414d-82a4-c9f79e962811 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:03.723937 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-2dbd2fc3-6f1c-414d-82a4-c9f79e962811 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:03.734846 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbd2fc3-6f1c-414d-82a4-c9f79e962811 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:03.737090 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbd2fc3-6f1c-414d-82a4-c9f79e962811 service neutron] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:03.749979 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1235/2473] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:07:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:07:13.348110 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=60 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:13.742982 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=30 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:18.491727 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bd9d0e-f88f-4608-b38e-191df23afc09 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:18.507392 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bd9d0e-f88f-4608-b38e-191df23afc09 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:18.520011 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf356ee6-e29c-4300-9b58-b7384208f407 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17da357e555420799ee681bffbcc268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:18.520011 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf356ee6-e29c-4300-9b58-b7384208f407 service nova] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:18.520011 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf356ee6-e29c-4300-9b58-b7384208f407 service nova] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:18.520011 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf356ee6-e29c-4300-9b58-b7384208f407 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:18.520740 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-bf356ee6-e29c-4300-9b58-b7384208f407 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:18.536904 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf356ee6-e29c-4300-9b58-b7384208f407 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:18.545649 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf356ee6-e29c-4300-9b58-b7384208f407 service nova] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:18.558945 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1239/2474] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:07:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:07:20.923501 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f94b78f-9378-4fed-9989-1f1c4625500f None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:20.924095 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f94b78f-9378-4fed-9989-1f1c4625500f None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:20.924095 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f94b78f-9378-4fed-9989-1f1c4625500f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:20.944356 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-4f94b78f-9378-4fed-9989-1f1c4625500f None None] MFA Rules not processed for user `f652a368b1cc42ae8b6f6c7a6dff6f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:20.945627 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-4f94b78f-9378-4fed-9989-1f1c4625500f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:20.953217 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1236/2475] 10.222.0.55 () {58 vars in 967 bytes} [Wed Aug 24 09:07:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:07:20.966585 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf356ee6-e29c-4300-9b58-b7384208f407 service nova] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:20.978964 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf356ee6-e29c-4300-9b58-b7384208f407 service nova] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:20.984881 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05b2ea4-d1e9-4289-95ef-c342f4173fbf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0ebb5ccabef45378f0c1da69721e051', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:20.985510 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05b2ea4-d1e9-4289-95ef-c342f4173fbf service swift] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:20.985792 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05b2ea4-d1e9-4289-95ef-c342f4173fbf service swift] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:20.986064 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05b2ea4-d1e9-4289-95ef-c342f4173fbf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:20.997270 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05b2ea4-d1e9-4289-95ef-c342f4173fbf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:20.998898 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05b2ea4-d1e9-4289-95ef-c342f4173fbf service swift] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:21.008457 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1240/2476] 10.222.0.55 () {60 vars in 1384 bytes} [Wed Aug 24 09:07:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 09:07:24.513522 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f94b78f-9378-4fed-9989-1f1c4625500f None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:24.523762 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f94b78f-9378-4fed-9989-1f1c4625500f None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:24.527475 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45f35d1-b6bc-473f-a04a-3f0b7cd3e66c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:24.528078 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45f35d1-b6bc-473f-a04a-3f0b7cd3e66c service neutron] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:24.528347 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45f35d1-b6bc-473f-a04a-3f0b7cd3e66c service neutron] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:24.528644 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45f35d1-b6bc-473f-a04a-3f0b7cd3e66c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:24.530455 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-a45f35d1-b6bc-473f-a04a-3f0b7cd3e66c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:24.592725 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45f35d1-b6bc-473f-a04a-3f0b7cd3e66c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:24.594227 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45f35d1-b6bc-473f-a04a-3f0b7cd3e66c service neutron] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:24.603944 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1237/2477] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:07:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:07:28.414582 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05b2ea4-d1e9-4289-95ef-c342f4173fbf service swift] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:28.416100 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-a05b2ea4-d1e9-4289-95ef-c342f4173fbf service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:28.432055 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05b2ea4-d1e9-4289-95ef-c342f4173fbf service swift] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:28.438083 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8c6ca5-762d-4726-b794-79d72c024692 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf0c99771661413aaa0f237af6b394ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:28.445104 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8c6ca5-762d-4726-b794-79d72c024692 service placement] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:28.445104 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8c6ca5-762d-4726-b794-79d72c024692 service placement] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:28.445104 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8c6ca5-762d-4726-b794-79d72c024692 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:28.445104 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-4b8c6ca5-762d-4726-b794-79d72c024692 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:28.453948 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8c6ca5-762d-4726-b794-79d72c024692 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:28.455611 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8c6ca5-762d-4726-b794-79d72c024692 service placement] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:28.466119 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1241/2478] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:07:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:07:28.557397 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45f35d1-b6bc-473f-a04a-3f0b7cd3e66c service neutron] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:28.558880 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-a45f35d1-b6bc-473f-a04a-3f0b7cd3e66c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:28.572197 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45f35d1-b6bc-473f-a04a-3f0b7cd3e66c service neutron] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:28.577883 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eea1c85-b889-46cd-888b-fe4153c0ef4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:28.578522 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eea1c85-b889-46cd-888b-fe4153c0ef4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:28.578805 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eea1c85-b889-46cd-888b-fe4153c0ef4a admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:28.579083 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eea1c85-b889-46cd-888b-fe4153c0ef4a admin admin] PATH_INFO: `/v3/users/f5d7aee76519492e81b0048edebba36f` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:28.580267 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eea1c85-b889-46cd-888b-fe4153c0ef4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5d7aee76519492e81b0048edebba36f)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:28.582228 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eea1c85-b889-46cd-888b-fe4153c0ef4a admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:28.931035 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-0eea1c85-b889-46cd-888b-fe4153c0ef4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5d7aee76519492e81b0048edebba36f'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:28.931718 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-0eea1c85-b889-46cd-888b-fe4153c0ef4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5d7aee76519492e81b0048edebba36f'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:28.932337 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-0eea1c85-b889-46cd-888b-fe4153c0ef4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5d7aee76519492e81b0048edebba36f'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:28.953866 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-0eea1c85-b889-46cd-888b-fe4153c0ef4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5d7aee76519492e81b0048edebba36f'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:29.087502 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-0eea1c85-b889-46cd-888b-fe4153c0ef4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5d7aee76519492e81b0048edebba36f'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:29.102117 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1238/2479] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 09:07:28 2022] DELETE /identity/v3/users/f5d7aee76519492e81b0048edebba36f => generated 0 bytes in 545 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:07:29.341649 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8c6ca5-762d-4726-b794-79d72c024692 service placement] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:29.362088 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8c6ca5-762d-4726-b794-79d72c024692 service placement] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:29.442617 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11a74e4-075e-4f6d-869c-d1225bc500af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:29.443287 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11a74e4-075e-4f6d-869c-d1225bc500af admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:29.443444 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11a74e4-075e-4f6d-869c-d1225bc500af admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:29.443721 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11a74e4-075e-4f6d-869c-d1225bc500af admin admin] PATH_INFO: `/v3/projects/02d5c74d88a24eb9814f49a6ff4eca56` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:29.444949 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11a74e4-075e-4f6d-869c-d1225bc500af admin admin] RBAC: Authorizing `identity:delete_project(project_id=02d5c74d88a24eb9814f49a6ff4eca56)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:29.446971 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11a74e4-075e-4f6d-869c-d1225bc500af admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:29.558675 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-a11a74e4-075e-4f6d-869c-d1225bc500af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02d5c74d88a24eb9814f49a6ff4eca56', 'request_id': 'req-a11a74e4-075e-4f6d-869c-d1225bc500af'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:29.564810 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-a11a74e4-075e-4f6d-869c-d1225bc500af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02d5c74d88a24eb9814f49a6ff4eca56', 'request_id': 'req-a11a74e4-075e-4f6d-869c-d1225bc500af'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:29.630201 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-a11a74e4-075e-4f6d-869c-d1225bc500af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02d5c74d88a24eb9814f49a6ff4eca56', 'request_id': 'req-a11a74e4-075e-4f6d-869c-d1225bc500af'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:29.643368 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-a11a74e4-075e-4f6d-869c-d1225bc500af admin admin] The token cache is being invalidate because project 02d5c74d88a24eb9814f49a6ff4eca56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:07:29.643655 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-a11a74e4-075e-4f6d-869c-d1225bc500af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:29.644300 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-a11a74e4-075e-4f6d-869c-d1225bc500af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:29.645794 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1242/2480] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 09:07:29 2022] DELETE /identity/v3/projects/02d5c74d88a24eb9814f49a6ff4eca56 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:07:29.650922 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aadfea-119d-441e-8a34-4266618f9eba None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:29.651440 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aadfea-119d-441e-8a34-4266618f9eba None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:29.651541 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aadfea-119d-441e-8a34-4266618f9eba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:29.672776 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-07aadfea-119d-441e-8a34-4266618f9eba None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:29.674028 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-07aadfea-119d-441e-8a34-4266618f9eba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:29.735782 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:29.744970 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1239/2481] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:07:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:07:29.754189 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2e36b2-079f-4510-b785-1f86f4e726ce None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:29.754467 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2e36b2-079f-4510-b785-1f86f4e726ce None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:29.754739 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2e36b2-079f-4510-b785-1f86f4e726ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:29.776057 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-8f2e36b2-079f-4510-b785-1f86f4e726ce None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:29.777502 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-8f2e36b2-079f-4510-b785-1f86f4e726ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:29.785940 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1243/2482] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:07:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:07:29.790547 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aadfea-119d-441e-8a34-4266618f9eba None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:29.801688 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aadfea-119d-441e-8a34-4266618f9eba None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:29.806395 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fc6c92-5bd3-4d3c-b013-a181583c5ae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:29.807021 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fc6c92-5bd3-4d3c-b013-a181583c5ae6 admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:29.807295 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fc6c92-5bd3-4d3c-b013-a181583c5ae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:29.807568 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fc6c92-5bd3-4d3c-b013-a181583c5ae6 admin admin] PATH_INFO: `/v3/domains` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:29.808220 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fc6c92-5bd3-4d3c-b013-a181583c5ae6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:07:29.808523 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fc6c92-5bd3-4d3c-b013-a181583c5ae6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:29.810163 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fc6c92-5bd3-4d3c-b013-a181583c5ae6 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:29.824472 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1240/2483] 10.222.0.55 () {58 vars in 1153 bytes} [Wed Aug 24 09:07:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:29.828994 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2e36b2-079f-4510-b785-1f86f4e726ce None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:29.840503 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2e36b2-079f-4510-b785-1f86f4e726ce None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:29.845615 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b586eabe-aa85-4e0e-bb96-e083fe4a1917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:29.846292 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b586eabe-aa85-4e0e-bb96-e083fe4a1917 admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:29.846660 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b586eabe-aa85-4e0e-bb96-e083fe4a1917 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:29.846914 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b586eabe-aa85-4e0e-bb96-e083fe4a1917 admin admin] PATH_INFO: `/v3/projects` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:29.847586 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b586eabe-aa85-4e0e-bb96-e083fe4a1917 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:29.849542 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b586eabe-aa85-4e0e-bb96-e083fe4a1917 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:29.998403 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1244/2484] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 09:07:29 2022] POST /identity/v3/projects => generated 367 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:07:30.003714 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fc6c92-5bd3-4d3c-b013-a181583c5ae6 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:30.020860 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fc6c92-5bd3-4d3c-b013-a181583c5ae6 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:30.087856 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e710676-18b1-43fe-bd26-ad042bad6e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:30.088479 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e710676-18b1-43fe-bd26-ad042bad6e77 admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:30.088787 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e710676-18b1-43fe-bd26-ad042bad6e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:30.089134 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e710676-18b1-43fe-bd26-ad042bad6e77 admin admin] PATH_INFO: `/v3/users` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:30.089995 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e710676-18b1-43fe-bd26-ad042bad6e77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:30.092032 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e710676-18b1-43fe-bd26-ad042bad6e77 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:30.203863 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1241/2485] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 09:07:30 2022] POST /identity/v3/users => generated 284 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:07:30.207985 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b586eabe-aa85-4e0e-bb96-e083fe4a1917 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:30.218713 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b586eabe-aa85-4e0e-bb96-e083fe4a1917 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:30.223731 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d7d5f2-d994-4939-bf6e-8fa4c12e76e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:30.224336 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d7d5f2-d994-4939-bf6e-8fa4c12e76e3 admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:30.224646 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d7d5f2-d994-4939-bf6e-8fa4c12e76e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:30.224934 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d7d5f2-d994-4939-bf6e-8fa4c12e76e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:30.225693 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d7d5f2-d994-4939-bf6e-8fa4c12e76e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:30.227298 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d7d5f2-d994-4939-bf6e-8fa4c12e76e3 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:30.238303 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1245/2486] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:07:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:07:30.242488 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e710676-18b1-43fe-bd26-ad042bad6e77 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:30.254267 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e710676-18b1-43fe-bd26-ad042bad6e77 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:30.258936 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4412fe48-a885-42c0-98d0-b9a7af04a09f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:30.259542 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4412fe48-a885-42c0-98d0-b9a7af04a09f admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:30.259824 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4412fe48-a885-42c0-98d0-b9a7af04a09f admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:30.260091 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4412fe48-a885-42c0-98d0-b9a7af04a09f admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:30.260761 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4412fe48-a885-42c0-98d0-b9a7af04a09f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:30.262520 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4412fe48-a885-42c0-98d0-b9a7af04a09f admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:30.273654 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1242/2487] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:07:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:07:30.277964 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d7d5f2-d994-4939-bf6e-8fa4c12e76e3 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:30.288411 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d7d5f2-d994-4939-bf6e-8fa4c12e76e3 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:30.303404 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76436ddf-1001-48e3-a563-c87e03b8e668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:30.303404 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76436ddf-1001-48e3-a563-c87e03b8e668 admin admin] REQUEST_METHOD: `PUT` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:30.303404 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76436ddf-1001-48e3-a563-c87e03b8e668 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:30.303404 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76436ddf-1001-48e3-a563-c87e03b8e668 admin admin] PATH_INFO: `/v3/projects/2d09daa18d3446ddb845be7e7f91b925/users/c25d626a154542e8926c3aeaa9c201ca/roles/b674fd0a592e4e0cada811bba5ceea4d` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:30.303975 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:30.309670 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76436ddf-1001-48e3-a563-c87e03b8e668 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d09daa18d3446ddb845be7e7f91b925, user_id=c25d626a154542e8926c3aeaa9c201ca, role_id=b674fd0a592e4e0cada811bba5ceea4d)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:30.313035 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76436ddf-1001-48e3-a563-c87e03b8e668 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:30.397553 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1246/2488] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 09:07:30 2022] PUT /identity/v3/projects/2d09daa18d3446ddb845be7e7f91b925/users/c25d626a154542e8926c3aeaa9c201ca/roles/b674fd0a592e4e0cada811bba5ceea4d => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:07:30.407997 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103b8b7c-111e-4a87-b017-af9e5dfd649e None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:30.408362 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103b8b7c-111e-4a87-b017-af9e5dfd649e None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:30.408666 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103b8b7c-111e-4a87-b017-af9e5dfd649e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:30.439703 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-103b8b7c-111e-4a87-b017-af9e5dfd649e None None] MFA Rules not processed for user `c25d626a154542e8926c3aeaa9c201ca`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:30.441083 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-103b8b7c-111e-4a87-b017-af9e5dfd649e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:30.510425 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1243/2489] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:07:30 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:07:30.521251 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76436ddf-1001-48e3-a563-c87e03b8e668 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:30.523069 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-76436ddf-1001-48e3-a563-c87e03b8e668 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:30.583256 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76436ddf-1001-48e3-a563-c87e03b8e668 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:30.587045 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6908f63-6d97-422c-a94d-6716cbc39856 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbb9ac667f04c35a63b2689df18efed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:30.587656 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6908f63-6d97-422c-a94d-6716cbc39856 service cinder] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:30.587933 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6908f63-6d97-422c-a94d-6716cbc39856 service cinder] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:30.588203 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6908f63-6d97-422c-a94d-6716cbc39856 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:30.597506 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6908f63-6d97-422c-a94d-6716cbc39856 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:30.598922 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6908f63-6d97-422c-a94d-6716cbc39856 service cinder] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:30.609026 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1247/2490] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:07:30 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:07:36.912384 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103b8b7c-111e-4a87-b017-af9e5dfd649e None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:36.913806 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-103b8b7c-111e-4a87-b017-af9e5dfd649e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:36.978018 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103b8b7c-111e-4a87-b017-af9e5dfd649e None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:36.981799 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c38275-577a-44d3-b124-a3125cc4e508 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:36.982365 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c38275-577a-44d3-b124-a3125cc4e508 service neutron] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:36.982669 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c38275-577a-44d3-b124-a3125cc4e508 service neutron] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:36.982919 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c38275-577a-44d3-b124-a3125cc4e508 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:36.984753 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-80c38275-577a-44d3-b124-a3125cc4e508 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:36.994929 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:37.052894 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c38275-577a-44d3-b124-a3125cc4e508 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:37.054381 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c38275-577a-44d3-b124-a3125cc4e508 service neutron] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:37.064747 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1244/2491] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:07:36 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:07:40.603873 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=31,keystone:INSERT=1 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:43.842279 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6908f63-6d97-422c-a94d-6716cbc39856 service cinder] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:43.913258 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6908f63-6d97-422c-a94d-6716cbc39856 service cinder] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:43.918165 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975c954c-e35f-40c7-b60c-faa2226df9b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:43.918799 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975c954c-e35f-40c7-b60c-faa2226df9b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:43.919076 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975c954c-e35f-40c7-b60c-faa2226df9b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:43.919339 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975c954c-e35f-40c7-b60c-faa2226df9b3 admin admin] PATH_INFO: `/v3/users/c25d626a154542e8926c3aeaa9c201ca` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:43.920507 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975c954c-e35f-40c7-b60c-faa2226df9b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c25d626a154542e8926c3aeaa9c201ca)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:43.923027 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975c954c-e35f-40c7-b60c-faa2226df9b3 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:44.101937 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-975c954c-e35f-40c7-b60c-faa2226df9b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c25d626a154542e8926c3aeaa9c201ca'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:44.102530 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-975c954c-e35f-40c7-b60c-faa2226df9b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c25d626a154542e8926c3aeaa9c201ca'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:44.103176 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-975c954c-e35f-40c7-b60c-faa2226df9b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c25d626a154542e8926c3aeaa9c201ca'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:44.115893 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-975c954c-e35f-40c7-b60c-faa2226df9b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c25d626a154542e8926c3aeaa9c201ca'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:44.202677 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-975c954c-e35f-40c7-b60c-faa2226df9b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c25d626a154542e8926c3aeaa9c201ca'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:44.217564 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1248/2492] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 09:07:43 2022] DELETE /identity/v3/users/c25d626a154542e8926c3aeaa9c201ca => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:07:44.222733 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846a181d-518c-4015-90ad-29630e353e2b None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:44.223281 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846a181d-518c-4015-90ad-29630e353e2b None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:44.223281 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846a181d-518c-4015-90ad-29630e353e2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:44.245174 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-846a181d-518c-4015-90ad-29630e353e2b None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:44.246472 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-846a181d-518c-4015-90ad-29630e353e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:44.310702 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1245/2493] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:07:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:07:44.322879 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975c954c-e35f-40c7-b60c-faa2226df9b3 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:44.324578 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-975c954c-e35f-40c7-b60c-faa2226df9b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:44.397220 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975c954c-e35f-40c7-b60c-faa2226df9b3 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:44.400926 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e4cc5c-6cef-44b7-a614-4e5513c64c2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:44.401563 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e4cc5c-6cef-44b7-a614-4e5513c64c2c service neutron] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:44.401826 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e4cc5c-6cef-44b7-a614-4e5513c64c2c service neutron] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:44.402090 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e4cc5c-6cef-44b7-a614-4e5513c64c2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:44.413421 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e4cc5c-6cef-44b7-a614-4e5513c64c2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:44.414855 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e4cc5c-6cef-44b7-a614-4e5513c64c2c service neutron] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:44.425068 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1249/2494] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:07:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:07:44.819471 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846a181d-518c-4015-90ad-29630e353e2b None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:44.821786 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-846a181d-518c-4015-90ad-29630e353e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:44.843359 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846a181d-518c-4015-90ad-29630e353e2b None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:44.849181 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c7677d-258b-4984-a5e9-2690b7352aab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:44.850338 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c7677d-258b-4984-a5e9-2690b7352aab admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:44.850338 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c7677d-258b-4984-a5e9-2690b7352aab admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:44.850575 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c7677d-258b-4984-a5e9-2690b7352aab admin admin] PATH_INFO: `/v3/projects/2d09daa18d3446ddb845be7e7f91b925` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:44.851917 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c7677d-258b-4984-a5e9-2690b7352aab admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d09daa18d3446ddb845be7e7f91b925)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:44.853891 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c7677d-258b-4984-a5e9-2690b7352aab admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:44.957166 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-63c7677d-258b-4984-a5e9-2690b7352aab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d09daa18d3446ddb845be7e7f91b925', 'request_id': 'req-63c7677d-258b-4984-a5e9-2690b7352aab'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:45.024821 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-63c7677d-258b-4984-a5e9-2690b7352aab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d09daa18d3446ddb845be7e7f91b925', 'request_id': 'req-63c7677d-258b-4984-a5e9-2690b7352aab'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:45.032407 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-63c7677d-258b-4984-a5e9-2690b7352aab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d09daa18d3446ddb845be7e7f91b925', 'request_id': 'req-63c7677d-258b-4984-a5e9-2690b7352aab'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:45.043224 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-63c7677d-258b-4984-a5e9-2690b7352aab admin admin] The token cache is being invalidate because project 2d09daa18d3446ddb845be7e7f91b925 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:07:45.043510 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-63c7677d-258b-4984-a5e9-2690b7352aab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:45.044153 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-63c7677d-258b-4984-a5e9-2690b7352aab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:45.045718 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1246/2495] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 09:07:44 2022] DELETE /identity/v3/projects/2d09daa18d3446ddb845be7e7f91b925 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:07:45.051873 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf39c51e-62fc-4b5f-9666-2ca382ea0d23 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:45.052355 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf39c51e-62fc-4b5f-9666-2ca382ea0d23 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:45.052490 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf39c51e-62fc-4b5f-9666-2ca382ea0d23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:45.075851 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-cf39c51e-62fc-4b5f-9666-2ca382ea0d23 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:45.077117 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-cf39c51e-62fc-4b5f-9666-2ca382ea0d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:45.097126 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:45.140404 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1250/2496] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:07:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:07:45.149241 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f84365-396d-4b8a-a605-145996cca86a None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:45.149521 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f84365-396d-4b8a-a605-145996cca86a None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:45.149798 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f84365-396d-4b8a-a605-145996cca86a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:45.171067 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-15f84365-396d-4b8a-a605-145996cca86a None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:45.172809 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-15f84365-396d-4b8a-a605-145996cca86a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:45.181292 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1247/2497] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:07:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:07:45.185701 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf39c51e-62fc-4b5f-9666-2ca382ea0d23 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:45.197445 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf39c51e-62fc-4b5f-9666-2ca382ea0d23 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:45.202986 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d446c02a-dec5-4bca-bd1c-686d7d217949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:45.204172 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d446c02a-dec5-4bca-bd1c-686d7d217949 admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:45.204453 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d446c02a-dec5-4bca-bd1c-686d7d217949 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:45.204770 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d446c02a-dec5-4bca-bd1c-686d7d217949 admin admin] PATH_INFO: `/v3/domains` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:45.205435 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d446c02a-dec5-4bca-bd1c-686d7d217949 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:07:45.205818 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d446c02a-dec5-4bca-bd1c-686d7d217949 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:45.207441 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d446c02a-dec5-4bca-bd1c-686d7d217949 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:45.221529 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1251/2498] 10.222.0.55 () {58 vars in 1153 bytes} [Wed Aug 24 09:07:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:45.226009 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f84365-396d-4b8a-a605-145996cca86a None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:45.236050 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f84365-396d-4b8a-a605-145996cca86a None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:45.240954 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24655972-6a7e-49fd-b4dd-dd97c416d65a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:45.241560 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24655972-6a7e-49fd-b4dd-dd97c416d65a admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:45.241837 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24655972-6a7e-49fd-b4dd-dd97c416d65a admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:45.242108 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24655972-6a7e-49fd-b4dd-dd97c416d65a admin admin] PATH_INFO: `/v3/projects` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:45.242734 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24655972-6a7e-49fd-b4dd-dd97c416d65a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:45.244766 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24655972-6a7e-49fd-b4dd-dd97c416d65a admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:45.329177 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:45.335534 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1248/2499] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 09:07:45 2022] POST /identity/v3/projects => generated 387 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:07:45.339800 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d446c02a-dec5-4bca-bd1c-686d7d217949 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:45.350555 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d446c02a-dec5-4bca-bd1c-686d7d217949 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:45.407853 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4939aa59-6be3-4f11-8402-20f16d5bae9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:45.408491 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4939aa59-6be3-4f11-8402-20f16d5bae9f admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:45.408805 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4939aa59-6be3-4f11-8402-20f16d5bae9f admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:45.409077 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4939aa59-6be3-4f11-8402-20f16d5bae9f admin admin] PATH_INFO: `/v3/users` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:45.409960 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4939aa59-6be3-4f11-8402-20f16d5bae9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:45.411891 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4939aa59-6be3-4f11-8402-20f16d5bae9f admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:45.494636 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1252/2500] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 09:07:45 2022] POST /identity/v3/users => generated 294 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:07:45.498947 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24655972-6a7e-49fd-b4dd-dd97c416d65a admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:45.510739 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24655972-6a7e-49fd-b4dd-dd97c416d65a admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:45.517075 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffcf9a7-4b84-4c03-8155-ca065d8952b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:45.517695 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffcf9a7-4b84-4c03-8155-ca065d8952b7 admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:45.517998 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffcf9a7-4b84-4c03-8155-ca065d8952b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:45.518298 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffcf9a7-4b84-4c03-8155-ca065d8952b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:45.518915 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffcf9a7-4b84-4c03-8155-ca065d8952b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:45.520525 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffcf9a7-4b84-4c03-8155-ca065d8952b7 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:45.536300 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1249/2501] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:07:45 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:07:45.540656 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4939aa59-6be3-4f11-8402-20f16d5bae9f admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:45.551900 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4939aa59-6be3-4f11-8402-20f16d5bae9f admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:45.556886 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96de763-eddc-41cb-8b63-198bc9be4573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:45.557596 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96de763-eddc-41cb-8b63-198bc9be4573 admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:45.558021 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96de763-eddc-41cb-8b63-198bc9be4573 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:45.558388 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96de763-eddc-41cb-8b63-198bc9be4573 admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:45.559251 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96de763-eddc-41cb-8b63-198bc9be4573 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:45.561137 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96de763-eddc-41cb-8b63-198bc9be4573 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:45.573050 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1253/2502] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:07:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:07:45.577987 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffcf9a7-4b84-4c03-8155-ca065d8952b7 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:45.589264 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffcf9a7-4b84-4c03-8155-ca065d8952b7 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:45.594576 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a297110-7e8d-4c42-814f-e95cd9187ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:45.595256 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a297110-7e8d-4c42-814f-e95cd9187ac3 admin admin] REQUEST_METHOD: `PUT` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:45.595537 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a297110-7e8d-4c42-814f-e95cd9187ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:45.595822 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a297110-7e8d-4c42-814f-e95cd9187ac3 admin admin] PATH_INFO: `/v3/projects/e7743d9babd543e1bbfe3201b20a109d/users/7e8c63a2c77f4b6baf3180b92070e199/roles/b674fd0a592e4e0cada811bba5ceea4d` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:45.614562 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a297110-7e8d-4c42-814f-e95cd9187ac3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7743d9babd543e1bbfe3201b20a109d, user_id=7e8c63a2c77f4b6baf3180b92070e199, role_id=b674fd0a592e4e0cada811bba5ceea4d)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:45.618528 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a297110-7e8d-4c42-814f-e95cd9187ac3 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:45.699820 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1250/2503] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 09:07:45 2022] PUT /identity/v3/projects/e7743d9babd543e1bbfe3201b20a109d/users/7e8c63a2c77f4b6baf3180b92070e199/roles/b674fd0a592e4e0cada811bba5ceea4d => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:07:45.712697 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb305ba-d799-4743-a9c4-71632a323962 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:45.716436 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb305ba-d799-4743-a9c4-71632a323962 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:45.716436 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb305ba-d799-4743-a9c4-71632a323962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:45.743431 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-3fb305ba-d799-4743-a9c4-71632a323962 None None] MFA Rules not processed for user `7e8c63a2c77f4b6baf3180b92070e199`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:45.744719 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-3fb305ba-d799-4743-a9c4-71632a323962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:45.811632 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1254/2504] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:07:45 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:07:45.816865 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a297110-7e8d-4c42-814f-e95cd9187ac3 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:45.877747 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a297110-7e8d-4c42-814f-e95cd9187ac3 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:45.882434 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc7c267-2b63-4a5b-9714-45e2effd08c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:45.883028 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc7c267-2b63-4a5b-9714-45e2effd08c7 admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:45.883306 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc7c267-2b63-4a5b-9714-45e2effd08c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:45.883592 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc7c267-2b63-4a5b-9714-45e2effd08c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:45.884225 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc7c267-2b63-4a5b-9714-45e2effd08c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:45.886145 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc7c267-2b63-4a5b-9714-45e2effd08c7 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:45.952109 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1251/2505] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 09:07:45 2022] POST /identity/v3/projects => generated 383 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:07:45.956169 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb305ba-d799-4743-a9c4-71632a323962 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:45.967075 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb305ba-d799-4743-a9c4-71632a323962 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:45.979841 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:46.025109 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213bb949-1990-43e5-bfd4-597603d56ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:46.025711 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213bb949-1990-43e5-bfd4-597603d56ff7 admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:46.026032 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213bb949-1990-43e5-bfd4-597603d56ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:46.026412 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213bb949-1990-43e5-bfd4-597603d56ff7 admin admin] PATH_INFO: `/v3/users` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:46.027032 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213bb949-1990-43e5-bfd4-597603d56ff7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:46.029042 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213bb949-1990-43e5-bfd4-597603d56ff7 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:46.086762 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1255/2506] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 09:07:45 2022] POST /identity/v3/users => generated 292 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:07:46.090959 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc7c267-2b63-4a5b-9714-45e2effd08c7 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:46.108359 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc7c267-2b63-4a5b-9714-45e2effd08c7 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:46.113504 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e421c03b-9934-43c1-bb8f-529e65aa9656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:46.114098 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e421c03b-9934-43c1-bb8f-529e65aa9656 admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:46.114375 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e421c03b-9934-43c1-bb8f-529e65aa9656 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:46.114644 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e421c03b-9934-43c1-bb8f-529e65aa9656 admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:46.115280 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e421c03b-9934-43c1-bb8f-529e65aa9656 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:46.116993 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e421c03b-9934-43c1-bb8f-529e65aa9656 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:46.128031 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1252/2507] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:07:46 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:07:46.132075 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213bb949-1990-43e5-bfd4-597603d56ff7 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:46.142226 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213bb949-1990-43e5-bfd4-597603d56ff7 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:46.146740 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20594e68-b75a-4460-9d17-dc1535b65faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:46.147332 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20594e68-b75a-4460-9d17-dc1535b65faa admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:46.147609 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20594e68-b75a-4460-9d17-dc1535b65faa admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:46.147877 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20594e68-b75a-4460-9d17-dc1535b65faa admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:46.148510 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20594e68-b75a-4460-9d17-dc1535b65faa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:46.150129 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20594e68-b75a-4460-9d17-dc1535b65faa admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:46.160839 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1256/2508] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:07:46 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:07:46.164952 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e421c03b-9934-43c1-bb8f-529e65aa9656 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:46.176104 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e421c03b-9934-43c1-bb8f-529e65aa9656 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:46.180889 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86532648-8f01-42fe-a8cc-6c2eec03dc0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:46.181503 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86532648-8f01-42fe-a8cc-6c2eec03dc0b admin admin] REQUEST_METHOD: `PUT` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:46.181773 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86532648-8f01-42fe-a8cc-6c2eec03dc0b admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:46.182044 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86532648-8f01-42fe-a8cc-6c2eec03dc0b admin admin] PATH_INFO: `/v3/projects/473e58d9334e4e3bbef1aa8d507ec9a0/users/dc41162723bb49ebb291932e47a5484b/roles/b674fd0a592e4e0cada811bba5ceea4d` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:46.196709 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86532648-8f01-42fe-a8cc-6c2eec03dc0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=473e58d9334e4e3bbef1aa8d507ec9a0, user_id=dc41162723bb49ebb291932e47a5484b, role_id=b674fd0a592e4e0cada811bba5ceea4d)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:46.199348 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86532648-8f01-42fe-a8cc-6c2eec03dc0b admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:46.292699 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1253/2509] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 09:07:46 2022] PUT /identity/v3/projects/473e58d9334e4e3bbef1aa8d507ec9a0/users/dc41162723bb49ebb291932e47a5484b/roles/b674fd0a592e4e0cada811bba5ceea4d => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:07:46.303492 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920da97b-fe97-4b78-9826-f6ac4faec580 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:46.303779 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920da97b-fe97-4b78-9826-f6ac4faec580 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:46.307126 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920da97b-fe97-4b78-9826-f6ac4faec580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:46.331767 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-920da97b-fe97-4b78-9826-f6ac4faec580 None None] MFA Rules not processed for user `dc41162723bb49ebb291932e47a5484b`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:46.333041 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-920da97b-fe97-4b78-9826-f6ac4faec580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:46.396433 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1257/2510] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:07:46 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:07:46.401039 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86532648-8f01-42fe-a8cc-6c2eec03dc0b admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:46.463501 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86532648-8f01-42fe-a8cc-6c2eec03dc0b admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:46.468182 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79ad5ee-102a-434b-a70b-f0d459e84a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:46.468810 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79ad5ee-102a-434b-a70b-f0d459e84a48 admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:46.469580 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79ad5ee-102a-434b-a70b-f0d459e84a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:46.469851 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79ad5ee-102a-434b-a70b-f0d459e84a48 admin admin] PATH_INFO: `/v3/projects` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:46.470469 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79ad5ee-102a-434b-a70b-f0d459e84a48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:46.472363 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79ad5ee-102a-434b-a70b-f0d459e84a48 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:46.585771 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:46.592124 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1254/2511] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 09:07:46 2022] POST /identity/v3/projects => generated 385 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:07:46.597185 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920da97b-fe97-4b78-9826-f6ac4faec580 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:46.607857 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920da97b-fe97-4b78-9826-f6ac4faec580 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:46.674989 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea93d51-d237-462e-8673-b36dc6a3768e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:46.675598 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea93d51-d237-462e-8673-b36dc6a3768e admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:46.675876 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea93d51-d237-462e-8673-b36dc6a3768e admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:46.676148 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea93d51-d237-462e-8673-b36dc6a3768e admin admin] PATH_INFO: `/v3/users` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:46.676817 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea93d51-d237-462e-8673-b36dc6a3768e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:46.678748 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea93d51-d237-462e-8673-b36dc6a3768e admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:46.796161 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1258/2512] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 09:07:46 2022] POST /identity/v3/users => generated 299 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:07:46.800485 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79ad5ee-102a-434b-a70b-f0d459e84a48 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:46.811093 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79ad5ee-102a-434b-a70b-f0d459e84a48 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:46.815919 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624f1e97-bf67-4669-8b08-0b596fcd81a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:46.816522 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624f1e97-bf67-4669-8b08-0b596fcd81a5 admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:46.816831 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624f1e97-bf67-4669-8b08-0b596fcd81a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:46.817099 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624f1e97-bf67-4669-8b08-0b596fcd81a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:46.817738 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624f1e97-bf67-4669-8b08-0b596fcd81a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:46.819505 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624f1e97-bf67-4669-8b08-0b596fcd81a5 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:46.831071 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1255/2513] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:07:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:07:46.835379 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea93d51-d237-462e-8673-b36dc6a3768e admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:46.846910 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea93d51-d237-462e-8673-b36dc6a3768e admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:46.851737 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da38a697-f62a-4c7b-8434-f07b9350aced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:46.852441 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da38a697-f62a-4c7b-8434-f07b9350aced admin admin] REQUEST_METHOD: `PUT` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:46.852749 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da38a697-f62a-4c7b-8434-f07b9350aced admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:46.853022 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da38a697-f62a-4c7b-8434-f07b9350aced admin admin] PATH_INFO: `/v3/projects/3396dc20b2d840b883b808960e63d71c/users/3d3b1ec3741545c980525b560c6c79cc/roles/b870cd1acaa744e1ae1d58b9ac283821` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:46.865536 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:46.868813 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da38a697-f62a-4c7b-8434-f07b9350aced admin admin] RBAC: Authorizing `identity:create_grant(project_id=3396dc20b2d840b883b808960e63d71c, user_id=3d3b1ec3741545c980525b560c6c79cc, role_id=b870cd1acaa744e1ae1d58b9ac283821)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:46.871515 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da38a697-f62a-4c7b-8434-f07b9350aced admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:46.951069 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1259/2514] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 09:07:46 2022] PUT /identity/v3/projects/3396dc20b2d840b883b808960e63d71c/users/3d3b1ec3741545c980525b560c6c79cc/roles/b870cd1acaa744e1ae1d58b9ac283821 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:07:46.956028 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624f1e97-bf67-4669-8b08-0b596fcd81a5 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:46.966645 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624f1e97-bf67-4669-8b08-0b596fcd81a5 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:47.027694 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeb506f-f92e-4988-bd9a-2bcde16ae5f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:47.028311 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb506f-f92e-4988-bd9a-2bcde16ae5f4 admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:47.028662 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb506f-f92e-4988-bd9a-2bcde16ae5f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:47.028940 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb506f-f92e-4988-bd9a-2bcde16ae5f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:47.029587 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eeb506f-f92e-4988-bd9a-2bcde16ae5f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:47.031441 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eeb506f-f92e-4988-bd9a-2bcde16ae5f4 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:47.042660 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1256/2515] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:07:46 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:07:47.047321 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da38a697-f62a-4c7b-8434-f07b9350aced admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:47.058412 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da38a697-f62a-4c7b-8434-f07b9350aced admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:47.063201 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95adf809-3727-4bd6-a05b-9b541c076df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:47.064357 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95adf809-3727-4bd6-a05b-9b541c076df1 admin admin] REQUEST_METHOD: `PUT` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:47.064661 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95adf809-3727-4bd6-a05b-9b541c076df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:47.064938 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95adf809-3727-4bd6-a05b-9b541c076df1 admin admin] PATH_INFO: `/v3/projects/3396dc20b2d840b883b808960e63d71c/users/3d3b1ec3741545c980525b560c6c79cc/roles/b870cd1acaa744e1ae1d58b9ac283821` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:47.066863 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95adf809-3727-4bd6-a05b-9b541c076df1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3396dc20b2d840b883b808960e63d71c, user_id=3d3b1ec3741545c980525b560c6c79cc, role_id=b870cd1acaa744e1ae1d58b9ac283821)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:47.069555 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95adf809-3727-4bd6-a05b-9b541c076df1 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:47.080977 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1260/2516] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 09:07:47 2022] PUT /identity/v3/projects/3396dc20b2d840b883b808960e63d71c/users/3d3b1ec3741545c980525b560c6c79cc/roles/b870cd1acaa744e1ae1d58b9ac283821 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:07:47.092441 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aa19e0-62f0-463b-8b6e-8e25f2ea0d89 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:47.092753 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aa19e0-62f0-463b-8b6e-8e25f2ea0d89 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:47.093029 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aa19e0-62f0-463b-8b6e-8e25f2ea0d89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:47.122348 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-95aa19e0-62f0-463b-8b6e-8e25f2ea0d89 None None] MFA Rules not processed for user `3d3b1ec3741545c980525b560c6c79cc`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:47.123472 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-95aa19e0-62f0-463b-8b6e-8e25f2ea0d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:47.193814 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1257/2517] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:07:47 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:07:47.204493 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95adf809-3727-4bd6-a05b-9b541c076df1 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:47.206949 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-95adf809-3727-4bd6-a05b-9b541c076df1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:47.265578 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95adf809-3727-4bd6-a05b-9b541c076df1 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:47.269278 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b897b61-7e7b-4b6c-b947-5965fea30cc1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbb9ac667f04c35a63b2689df18efed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:47.269887 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b897b61-7e7b-4b6c-b947-5965fea30cc1 service cinder] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:47.270160 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b897b61-7e7b-4b6c-b947-5965fea30cc1 service cinder] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:47.270431 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b897b61-7e7b-4b6c-b947-5965fea30cc1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:47.279506 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b897b61-7e7b-4b6c-b947-5965fea30cc1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:47.280969 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b897b61-7e7b-4b6c-b947-5965fea30cc1 service cinder] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:47.290556 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1261/2518] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:07:47 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:07:48.495500 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aa19e0-62f0-463b-8b6e-8e25f2ea0d89 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:48.497060 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-95aa19e0-62f0-463b-8b6e-8e25f2ea0d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:48.509971 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aa19e0-62f0-463b-8b6e-8e25f2ea0d89 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:48.513586 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba184314-34e6-412b-91b2-3b0ce132aa61 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbb9ac667f04c35a63b2689df18efed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:48.514184 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba184314-34e6-412b-91b2-3b0ce132aa61 service cinder] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:48.514459 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba184314-34e6-412b-91b2-3b0ce132aa61 service cinder] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:48.514732 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba184314-34e6-412b-91b2-3b0ce132aa61 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:48.573243 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba184314-34e6-412b-91b2-3b0ce132aa61 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:48.574693 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba184314-34e6-412b-91b2-3b0ce132aa61 service cinder] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:48.584121 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1258/2519] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:07:48 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:07:55.840124 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b897b61-7e7b-4b6c-b947-5965fea30cc1 service cinder] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:55.848997 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b897b61-7e7b-4b6c-b947-5965fea30cc1 service cinder] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:55.852757 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e64cda-4212-476d-9b53-c609b8ec8301 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbb9ac667f04c35a63b2689df18efed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:55.853354 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e64cda-4212-476d-9b53-c609b8ec8301 service cinder] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:55.853629 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e64cda-4212-476d-9b53-c609b8ec8301 service cinder] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:55.853900 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e64cda-4212-476d-9b53-c609b8ec8301 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:55.913228 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e64cda-4212-476d-9b53-c609b8ec8301 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:55.914651 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e64cda-4212-476d-9b53-c609b8ec8301 service cinder] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:55.923608 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1262/2520] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:07:55 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 09:07:58.579552 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=96 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:00.158221 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba184314-34e6-412b-91b2-3b0ce132aa61 service cinder] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:00.222915 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba184314-34e6-412b-91b2-3b0ce132aa61 service cinder] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:00.228305 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40687b8-298c-432f-9a7a-eedc44d9e2e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:00.228967 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40687b8-298c-432f-9a7a-eedc44d9e2e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:00.229249 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40687b8-298c-432f-9a7a-eedc44d9e2e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:00.229525 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40687b8-298c-432f-9a7a-eedc44d9e2e1 admin admin] PATH_INFO: `/v3/users/7e8c63a2c77f4b6baf3180b92070e199` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:00.230735 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40687b8-298c-432f-9a7a-eedc44d9e2e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e8c63a2c77f4b6baf3180b92070e199)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:00.236715 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40687b8-298c-432f-9a7a-eedc44d9e2e1 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:00.374749 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-f40687b8-298c-432f-9a7a-eedc44d9e2e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e8c63a2c77f4b6baf3180b92070e199'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:00.375312 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-f40687b8-298c-432f-9a7a-eedc44d9e2e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e8c63a2c77f4b6baf3180b92070e199'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:00.375976 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-f40687b8-298c-432f-9a7a-eedc44d9e2e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e8c63a2c77f4b6baf3180b92070e199'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:00.388954 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-f40687b8-298c-432f-9a7a-eedc44d9e2e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e8c63a2c77f4b6baf3180b92070e199'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:00.458912 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-f40687b8-298c-432f-9a7a-eedc44d9e2e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e8c63a2c77f4b6baf3180b92070e199'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:00.471424 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1259/2521] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 09:08:00 2022] DELETE /identity/v3/users/7e8c63a2c77f4b6baf3180b92070e199 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:00.475923 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfccce89-993b-465c-ac2a-f914622491c8 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:00.477467 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfccce89-993b-465c-ac2a-f914622491c8 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:00.477467 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfccce89-993b-465c-ac2a-f914622491c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:00.495261 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-cfccce89-993b-465c-ac2a-f914622491c8 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:00.496460 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-cfccce89-993b-465c-ac2a-f914622491c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:00.556817 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1263/2522] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:08:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:00.567258 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40687b8-298c-432f-9a7a-eedc44d9e2e1 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:00.568726 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-f40687b8-298c-432f-9a7a-eedc44d9e2e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:00.632077 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40687b8-298c-432f-9a7a-eedc44d9e2e1 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:00.635877 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02034e4c-6e20-4f87-907c-378d886c6475 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:00.636477 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02034e4c-6e20-4f87-907c-378d886c6475 service neutron] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:00.636781 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02034e4c-6e20-4f87-907c-378d886c6475 service neutron] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:00.637051 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02034e4c-6e20-4f87-907c-378d886c6475 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:00.646746 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02034e4c-6e20-4f87-907c-378d886c6475 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:00.648153 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02034e4c-6e20-4f87-907c-378d886c6475 service neutron] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:00.658025 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1260/2523] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:08:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:08:01.113464 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfccce89-993b-465c-ac2a-f914622491c8 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:01.115998 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-cfccce89-993b-465c-ac2a-f914622491c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:01.129708 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfccce89-993b-465c-ac2a-f914622491c8 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:01.142160 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb20cc6-2740-46ba-8c10-c24a72ed45e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:01.142827 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb20cc6-2740-46ba-8c10-c24a72ed45e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:01.143108 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb20cc6-2740-46ba-8c10-c24a72ed45e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:01.143482 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb20cc6-2740-46ba-8c10-c24a72ed45e7 admin admin] PATH_INFO: `/v3/projects/e7743d9babd543e1bbfe3201b20a109d` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:01.144525 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb20cc6-2740-46ba-8c10-c24a72ed45e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7743d9babd543e1bbfe3201b20a109d)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:01.149063 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb20cc6-2740-46ba-8c10-c24a72ed45e7 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:01.167059 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:01.260035 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-deb20cc6-2740-46ba-8c10-c24a72ed45e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7743d9babd543e1bbfe3201b20a109d', 'request_id': 'req-deb20cc6-2740-46ba-8c10-c24a72ed45e7'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:01.265979 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-deb20cc6-2740-46ba-8c10-c24a72ed45e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7743d9babd543e1bbfe3201b20a109d', 'request_id': 'req-deb20cc6-2740-46ba-8c10-c24a72ed45e7'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:01.333094 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-deb20cc6-2740-46ba-8c10-c24a72ed45e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7743d9babd543e1bbfe3201b20a109d', 'request_id': 'req-deb20cc6-2740-46ba-8c10-c24a72ed45e7'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:01.346340 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-deb20cc6-2740-46ba-8c10-c24a72ed45e7 admin admin] The token cache is being invalidate because project e7743d9babd543e1bbfe3201b20a109d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:08:01.346635 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-deb20cc6-2740-46ba-8c10-c24a72ed45e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:01.347298 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-deb20cc6-2740-46ba-8c10-c24a72ed45e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:01.349428 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1264/2524] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 09:08:01 2022] DELETE /identity/v3/projects/e7743d9babd543e1bbfe3201b20a109d => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:01.353621 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02034e4c-6e20-4f87-907c-378d886c6475 service neutron] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:01.363549 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02034e4c-6e20-4f87-907c-378d886c6475 service neutron] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:01.383083 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:01.424746 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfabb69-cfb4-4e55-9472-5a95cd417bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:01.425299 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfabb69-cfb4-4e55-9472-5a95cd417bc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:01.425575 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfabb69-cfb4-4e55-9472-5a95cd417bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:01.425880 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfabb69-cfb4-4e55-9472-5a95cd417bc4 admin admin] PATH_INFO: `/v3/users/dc41162723bb49ebb291932e47a5484b` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:01.427227 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfabb69-cfb4-4e55-9472-5a95cd417bc4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc41162723bb49ebb291932e47a5484b)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:01.429277 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfabb69-cfb4-4e55-9472-5a95cd417bc4 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:01.592156 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-9dfabb69-cfb4-4e55-9472-5a95cd417bc4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc41162723bb49ebb291932e47a5484b'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:01.592832 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-9dfabb69-cfb4-4e55-9472-5a95cd417bc4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc41162723bb49ebb291932e47a5484b'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:01.593439 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-9dfabb69-cfb4-4e55-9472-5a95cd417bc4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc41162723bb49ebb291932e47a5484b'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:01.606865 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-9dfabb69-cfb4-4e55-9472-5a95cd417bc4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc41162723bb49ebb291932e47a5484b'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:01.670012 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-9dfabb69-cfb4-4e55-9472-5a95cd417bc4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc41162723bb49ebb291932e47a5484b'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:01.683742 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1261/2525] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 09:08:01 2022] DELETE /identity/v3/users/dc41162723bb49ebb291932e47a5484b => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:08:02.117455 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb20cc6-2740-46ba-8c10-c24a72ed45e7 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:02.119001 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-deb20cc6-2740-46ba-8c10-c24a72ed45e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:02.133002 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb20cc6-2740-46ba-8c10-c24a72ed45e7 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:02.194088 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92576a43-db32-4f16-9ebf-ca4c81ed4a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:02.194676 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92576a43-db32-4f16-9ebf-ca4c81ed4a9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:02.194957 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92576a43-db32-4f16-9ebf-ca4c81ed4a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:02.195227 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92576a43-db32-4f16-9ebf-ca4c81ed4a9e admin admin] PATH_INFO: `/v3/projects/473e58d9334e4e3bbef1aa8d507ec9a0` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:02.196527 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92576a43-db32-4f16-9ebf-ca4c81ed4a9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=473e58d9334e4e3bbef1aa8d507ec9a0)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:02.198455 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92576a43-db32-4f16-9ebf-ca4c81ed4a9e admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:02.310491 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-92576a43-db32-4f16-9ebf-ca4c81ed4a9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '473e58d9334e4e3bbef1aa8d507ec9a0', 'request_id': 'req-92576a43-db32-4f16-9ebf-ca4c81ed4a9e'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:02.316455 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-92576a43-db32-4f16-9ebf-ca4c81ed4a9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '473e58d9334e4e3bbef1aa8d507ec9a0', 'request_id': 'req-92576a43-db32-4f16-9ebf-ca4c81ed4a9e'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:02.401199 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-92576a43-db32-4f16-9ebf-ca4c81ed4a9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '473e58d9334e4e3bbef1aa8d507ec9a0', 'request_id': 'req-92576a43-db32-4f16-9ebf-ca4c81ed4a9e'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:02.413095 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-92576a43-db32-4f16-9ebf-ca4c81ed4a9e admin admin] The token cache is being invalidate because project 473e58d9334e4e3bbef1aa8d507ec9a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:08:02.413378 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-92576a43-db32-4f16-9ebf-ca4c81ed4a9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:02.414001 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-92576a43-db32-4f16-9ebf-ca4c81ed4a9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:02.415478 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1265/2526] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 09:08:02 2022] DELETE /identity/v3/projects/473e58d9334e4e3bbef1aa8d507ec9a0 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:02.419677 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfabb69-cfb4-4e55-9472-5a95cd417bc4 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:02.421156 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-9dfabb69-cfb4-4e55-9472-5a95cd417bc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:02.434717 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfabb69-cfb4-4e55-9472-5a95cd417bc4 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:02.493119 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74129ab-7124-4de0-897c-a4000bee4905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:02.493669 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74129ab-7124-4de0-897c-a4000bee4905 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:02.493949 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74129ab-7124-4de0-897c-a4000bee4905 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:02.494226 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74129ab-7124-4de0-897c-a4000bee4905 admin admin] PATH_INFO: `/v3/users/3d3b1ec3741545c980525b560c6c79cc` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:02.495369 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74129ab-7124-4de0-897c-a4000bee4905 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d3b1ec3741545c980525b560c6c79cc)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:02.497337 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74129ab-7124-4de0-897c-a4000bee4905 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:02.699395 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:02.701859 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-a74129ab-7124-4de0-897c-a4000bee4905 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d3b1ec3741545c980525b560c6c79cc'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:02.702531 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-a74129ab-7124-4de0-897c-a4000bee4905 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d3b1ec3741545c980525b560c6c79cc'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:02.703419 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-a74129ab-7124-4de0-897c-a4000bee4905 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d3b1ec3741545c980525b560c6c79cc'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:02.715152 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-a74129ab-7124-4de0-897c-a4000bee4905 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d3b1ec3741545c980525b560c6c79cc'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:02.791128 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-a74129ab-7124-4de0-897c-a4000bee4905 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d3b1ec3741545c980525b560c6c79cc'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:02.814097 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1262/2527] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 09:08:02 2022] DELETE /identity/v3/users/3d3b1ec3741545c980525b560c6c79cc => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:03.209457 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92576a43-db32-4f16-9ebf-ca4c81ed4a9e admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:03.210985 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-92576a43-db32-4f16-9ebf-ca4c81ed4a9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:03.226718 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92576a43-db32-4f16-9ebf-ca4c81ed4a9e admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:03.259423 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:03.285596 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f67bd9c-f2b2-4d4a-9fcd-6ef42e8c6813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:03.286227 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f67bd9c-f2b2-4d4a-9fcd-6ef42e8c6813 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:03.286702 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f67bd9c-f2b2-4d4a-9fcd-6ef42e8c6813 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:03.287188 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f67bd9c-f2b2-4d4a-9fcd-6ef42e8c6813 admin admin] PATH_INFO: `/v3/projects/3396dc20b2d840b883b808960e63d71c` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:03.288736 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f67bd9c-f2b2-4d4a-9fcd-6ef42e8c6813 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3396dc20b2d840b883b808960e63d71c)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:03.290689 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f67bd9c-f2b2-4d4a-9fcd-6ef42e8c6813 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:03.396860 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-0f67bd9c-f2b2-4d4a-9fcd-6ef42e8c6813 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3396dc20b2d840b883b808960e63d71c', 'request_id': 'req-0f67bd9c-f2b2-4d4a-9fcd-6ef42e8c6813'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:03.402133 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-0f67bd9c-f2b2-4d4a-9fcd-6ef42e8c6813 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3396dc20b2d840b883b808960e63d71c', 'request_id': 'req-0f67bd9c-f2b2-4d4a-9fcd-6ef42e8c6813'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:03.492532 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-0f67bd9c-f2b2-4d4a-9fcd-6ef42e8c6813 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3396dc20b2d840b883b808960e63d71c', 'request_id': 'req-0f67bd9c-f2b2-4d4a-9fcd-6ef42e8c6813'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:03.511501 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-0f67bd9c-f2b2-4d4a-9fcd-6ef42e8c6813 admin admin] The token cache is being invalidate because project 3396dc20b2d840b883b808960e63d71c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:08:03.511789 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-0f67bd9c-f2b2-4d4a-9fcd-6ef42e8c6813 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:03.512438 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-0f67bd9c-f2b2-4d4a-9fcd-6ef42e8c6813 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:03.513933 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1266/2528] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 09:08:03 2022] DELETE /identity/v3/projects/3396dc20b2d840b883b808960e63d71c => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:03.520274 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f43fdd-2275-4182-ae79-cde82cb01ed0 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:03.521506 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f43fdd-2275-4182-ae79-cde82cb01ed0 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:03.521651 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f43fdd-2275-4182-ae79-cde82cb01ed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:03.541256 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-d4f43fdd-2275-4182-ae79-cde82cb01ed0 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:03.542458 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-d4f43fdd-2275-4182-ae79-cde82cb01ed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:03.599898 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1263/2529] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:08:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:03.608511 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8049ac5-e1f3-465a-b9a9-91170c063268 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:03.608831 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8049ac5-e1f3-465a-b9a9-91170c063268 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:03.609190 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8049ac5-e1f3-465a-b9a9-91170c063268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:03.629675 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-f8049ac5-e1f3-465a-b9a9-91170c063268 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:03.631394 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-f8049ac5-e1f3-465a-b9a9-91170c063268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:03.639359 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1267/2530] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:08:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:03.643802 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f43fdd-2275-4182-ae79-cde82cb01ed0 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:03.654389 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f43fdd-2275-4182-ae79-cde82cb01ed0 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:03.658965 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe00bda-7751-4f7b-b7c7-6fe8b9c962c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:03.659562 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe00bda-7751-4f7b-b7c7-6fe8b9c962c6 admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:03.659844 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe00bda-7751-4f7b-b7c7-6fe8b9c962c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:03.660376 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe00bda-7751-4f7b-b7c7-6fe8b9c962c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:03.661055 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe00bda-7751-4f7b-b7c7-6fe8b9c962c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:03.661387 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe00bda-7751-4f7b-b7c7-6fe8b9c962c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:03.663021 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe00bda-7751-4f7b-b7c7-6fe8b9c962c6 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:03.674801 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1264/2531] 10.222.0.55 () {58 vars in 1153 bytes} [Wed Aug 24 09:08:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:03.679768 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8049ac5-e1f3-465a-b9a9-91170c063268 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:03.689183 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8049ac5-e1f3-465a-b9a9-91170c063268 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:03.694445 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf58635-34d0-4c9d-b0ff-7f7e35220fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:03.695066 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf58635-34d0-4c9d-b0ff-7f7e35220fef admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:03.695341 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf58635-34d0-4c9d-b0ff-7f7e35220fef admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:03.695609 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf58635-34d0-4c9d-b0ff-7f7e35220fef admin admin] PATH_INFO: `/v3/projects` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:03.696233 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf58635-34d0-4c9d-b0ff-7f7e35220fef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:03.698181 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf58635-34d0-4c9d-b0ff-7f7e35220fef admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:03.804325 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1268/2532] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 09:08:03 2022] POST /identity/v3/projects => generated 381 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:08:03.808402 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe00bda-7751-4f7b-b7c7-6fe8b9c962c6 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:03.819630 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe00bda-7751-4f7b-b7c7-6fe8b9c962c6 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:03.883829 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2aba16-37c7-4072-a28a-844605cadb72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:03.884443 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2aba16-37c7-4072-a28a-844605cadb72 admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:03.884753 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2aba16-37c7-4072-a28a-844605cadb72 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:03.885021 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2aba16-37c7-4072-a28a-844605cadb72 admin admin] PATH_INFO: `/v3/users` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:03.885650 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2aba16-37c7-4072-a28a-844605cadb72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:03.887582 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2aba16-37c7-4072-a28a-844605cadb72 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:03.996865 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1265/2533] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 09:08:03 2022] POST /identity/v3/users => generated 291 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:08:04.001178 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf58635-34d0-4c9d-b0ff-7f7e35220fef admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:04.012239 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf58635-34d0-4c9d-b0ff-7f7e35220fef admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:04.017364 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93009314-1dfe-45d7-afc7-6b65da5434ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:04.018015 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93009314-1dfe-45d7-afc7-6b65da5434ce admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:04.018298 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93009314-1dfe-45d7-afc7-6b65da5434ce admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:04.018574 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93009314-1dfe-45d7-afc7-6b65da5434ce admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:04.019271 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93009314-1dfe-45d7-afc7-6b65da5434ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:04.021008 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93009314-1dfe-45d7-afc7-6b65da5434ce admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:04.032518 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1269/2534] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:08:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:08:04.037413 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2aba16-37c7-4072-a28a-844605cadb72 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:04.047920 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2aba16-37c7-4072-a28a-844605cadb72 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:04.053442 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0844b7ec-cd05-42d9-aed6-c42260a65472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:04.054053 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0844b7ec-cd05-42d9-aed6-c42260a65472 admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:04.054336 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0844b7ec-cd05-42d9-aed6-c42260a65472 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:04.054611 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0844b7ec-cd05-42d9-aed6-c42260a65472 admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:04.055257 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0844b7ec-cd05-42d9-aed6-c42260a65472 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:04.056921 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0844b7ec-cd05-42d9-aed6-c42260a65472 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:04.067351 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1266/2535] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:08:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:08:04.071710 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93009314-1dfe-45d7-afc7-6b65da5434ce admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:04.082072 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93009314-1dfe-45d7-afc7-6b65da5434ce admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:04.087103 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aadc19f-3851-4e55-b192-cc2cbf869224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:04.087749 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aadc19f-3851-4e55-b192-cc2cbf869224 admin admin] REQUEST_METHOD: `PUT` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:04.088063 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aadc19f-3851-4e55-b192-cc2cbf869224 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:04.088353 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aadc19f-3851-4e55-b192-cc2cbf869224 admin admin] PATH_INFO: `/v3/projects/e51fee26fe4e46f78e808bd5a0b32006/users/53ccb1719f4e47e199df29005f34c143/roles/b674fd0a592e4e0cada811bba5ceea4d` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:04.103849 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aadc19f-3851-4e55-b192-cc2cbf869224 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e51fee26fe4e46f78e808bd5a0b32006, user_id=53ccb1719f4e47e199df29005f34c143, role_id=b674fd0a592e4e0cada811bba5ceea4d)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:04.106556 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aadc19f-3851-4e55-b192-cc2cbf869224 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:04.196512 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1270/2536] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 09:08:04 2022] PUT /identity/v3/projects/e51fee26fe4e46f78e808bd5a0b32006/users/53ccb1719f4e47e199df29005f34c143/roles/b674fd0a592e4e0cada811bba5ceea4d => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:04.208070 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560dba83-1dc9-4a95-a86e-b15dbf7d1c83 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:04.208393 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560dba83-1dc9-4a95-a86e-b15dbf7d1c83 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:04.208630 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560dba83-1dc9-4a95-a86e-b15dbf7d1c83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:04.237815 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-560dba83-1dc9-4a95-a86e-b15dbf7d1c83 None None] MFA Rules not processed for user `53ccb1719f4e47e199df29005f34c143`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:04.239074 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-560dba83-1dc9-4a95-a86e-b15dbf7d1c83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:04.252970 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:04.313819 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1267/2537] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:08:04 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:04.318478 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aadc19f-3851-4e55-b192-cc2cbf869224 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:04.382039 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aadc19f-3851-4e55-b192-cc2cbf869224 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:04.386818 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17592fe-1919-48bf-baa1-3161731d0fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:04.387568 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17592fe-1919-48bf-baa1-3161731d0fef admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:04.387705 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17592fe-1919-48bf-baa1-3161731d0fef admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:04.387967 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17592fe-1919-48bf-baa1-3161731d0fef admin admin] PATH_INFO: `/v3/projects` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:04.388641 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17592fe-1919-48bf-baa1-3161731d0fef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:04.390617 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17592fe-1919-48bf-baa1-3161731d0fef admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:04.395492 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:04.513213 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1271/2538] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 09:08:04 2022] POST /identity/v3/projects => generated 381 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:08:04.517383 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560dba83-1dc9-4a95-a86e-b15dbf7d1c83 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:04.528213 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560dba83-1dc9-4a95-a86e-b15dbf7d1c83 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:04.589047 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ec8617-16ef-4a73-a49e-1cd10855a31f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:04.589655 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ec8617-16ef-4a73-a49e-1cd10855a31f admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:04.589934 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ec8617-16ef-4a73-a49e-1cd10855a31f admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:04.590205 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ec8617-16ef-4a73-a49e-1cd10855a31f admin admin] PATH_INFO: `/v3/users` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:04.590820 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ec8617-16ef-4a73-a49e-1cd10855a31f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:04.592958 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ec8617-16ef-4a73-a49e-1cd10855a31f admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:04.710789 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1268/2539] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 09:08:04 2022] POST /identity/v3/users => generated 297 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:08:04.714899 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17592fe-1919-48bf-baa1-3161731d0fef admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:04.725585 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17592fe-1919-48bf-baa1-3161731d0fef admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:04.730269 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd268030-c491-42d3-aa80-6478896a11bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:04.730870 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd268030-c491-42d3-aa80-6478896a11bf admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:04.731145 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd268030-c491-42d3-aa80-6478896a11bf admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:04.731412 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd268030-c491-42d3-aa80-6478896a11bf admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:04.732076 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd268030-c491-42d3-aa80-6478896a11bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:04.733704 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd268030-c491-42d3-aa80-6478896a11bf admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:04.745323 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1272/2540] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:08:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:08:04.749925 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ec8617-16ef-4a73-a49e-1cd10855a31f admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:04.761061 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ec8617-16ef-4a73-a49e-1cd10855a31f admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:04.765816 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd9ce09-4221-4e83-9515-a104edcfd2ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:04.766409 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd9ce09-4221-4e83-9515-a104edcfd2ac admin admin] REQUEST_METHOD: `PUT` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:04.766686 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd9ce09-4221-4e83-9515-a104edcfd2ac admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:04.766955 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd9ce09-4221-4e83-9515-a104edcfd2ac admin admin] PATH_INFO: `/v3/projects/1f81b11b49184cbdac21b6a3a13ffe65/users/bd0b1e50bbc04ac28edf3fa1fb3a447a/roles/b870cd1acaa744e1ae1d58b9ac283821` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:04.782219 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd9ce09-4221-4e83-9515-a104edcfd2ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f81b11b49184cbdac21b6a3a13ffe65, user_id=bd0b1e50bbc04ac28edf3fa1fb3a447a, role_id=b870cd1acaa744e1ae1d58b9ac283821)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:04.784898 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd9ce09-4221-4e83-9515-a104edcfd2ac admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:04.848097 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1269/2541] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 09:08:04 2022] PUT /identity/v3/projects/1f81b11b49184cbdac21b6a3a13ffe65/users/bd0b1e50bbc04ac28edf3fa1fb3a447a/roles/b870cd1acaa744e1ae1d58b9ac283821 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:04.853404 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd268030-c491-42d3-aa80-6478896a11bf admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:04.863637 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd268030-c491-42d3-aa80-6478896a11bf admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:04.924165 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8c5360-ef3f-42ac-9bd9-1af957c1ac48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:04.924820 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8c5360-ef3f-42ac-9bd9-1af957c1ac48 admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:04.925100 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8c5360-ef3f-42ac-9bd9-1af957c1ac48 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:04.925380 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8c5360-ef3f-42ac-9bd9-1af957c1ac48 admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:04.926015 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8c5360-ef3f-42ac-9bd9-1af957c1ac48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:04.928191 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8c5360-ef3f-42ac-9bd9-1af957c1ac48 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:04.939048 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1273/2542] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:08:04 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:08:04.943272 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd9ce09-4221-4e83-9515-a104edcfd2ac admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:04.953876 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd9ce09-4221-4e83-9515-a104edcfd2ac admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:04.958707 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15ffc12-c5e2-4b63-9288-5583f69de34a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:04.959368 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15ffc12-c5e2-4b63-9288-5583f69de34a admin admin] REQUEST_METHOD: `PUT` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:04.959719 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15ffc12-c5e2-4b63-9288-5583f69de34a admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:04.960023 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15ffc12-c5e2-4b63-9288-5583f69de34a admin admin] PATH_INFO: `/v3/projects/1f81b11b49184cbdac21b6a3a13ffe65/users/bd0b1e50bbc04ac28edf3fa1fb3a447a/roles/b870cd1acaa744e1ae1d58b9ac283821` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:04.961886 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15ffc12-c5e2-4b63-9288-5583f69de34a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f81b11b49184cbdac21b6a3a13ffe65, user_id=bd0b1e50bbc04ac28edf3fa1fb3a447a, role_id=b870cd1acaa744e1ae1d58b9ac283821)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:04.964513 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15ffc12-c5e2-4b63-9288-5583f69de34a admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:04.975818 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1270/2543] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 09:08:04 2022] PUT /identity/v3/projects/1f81b11b49184cbdac21b6a3a13ffe65/users/bd0b1e50bbc04ac28edf3fa1fb3a447a/roles/b870cd1acaa744e1ae1d58b9ac283821 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:04.987071 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c87201-a8c8-48c2-9577-783819a5bc9d None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:04.987462 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c87201-a8c8-48c2-9577-783819a5bc9d None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:04.987745 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c87201-a8c8-48c2-9577-783819a5bc9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:05.020594 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-21c87201-a8c8-48c2-9577-783819a5bc9d None None] MFA Rules not processed for user `bd0b1e50bbc04ac28edf3fa1fb3a447a`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:05.021755 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-21c87201-a8c8-48c2-9577-783819a5bc9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:05.092442 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1274/2544] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:08:04 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:08:05.104874 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15ffc12-c5e2-4b63-9288-5583f69de34a admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:05.107010 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-c15ffc12-c5e2-4b63-9288-5583f69de34a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:05.170262 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15ffc12-c5e2-4b63-9288-5583f69de34a admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:05.174287 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29ea7e8-9fe4-461e-8b90-f1b8299f1421 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbb9ac667f04c35a63b2689df18efed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:05.174899 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29ea7e8-9fe4-461e-8b90-f1b8299f1421 service cinder] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:05.175174 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29ea7e8-9fe4-461e-8b90-f1b8299f1421 service cinder] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:05.175444 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29ea7e8-9fe4-461e-8b90-f1b8299f1421 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:05.185447 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29ea7e8-9fe4-461e-8b90-f1b8299f1421 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:05.186880 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29ea7e8-9fe4-461e-8b90-f1b8299f1421 service cinder] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:05.196978 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1271/2545] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:08:05 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:08:05.331028 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c87201-a8c8-48c2-9577-783819a5bc9d None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:05.344456 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c87201-a8c8-48c2-9577-783819a5bc9d None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:05.348239 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9435efd3-4154-4aa1-9ea9-cd305ef72622 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbb9ac667f04c35a63b2689df18efed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:05.349421 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9435efd3-4154-4aa1-9ea9-cd305ef72622 service cinder] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:05.349699 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9435efd3-4154-4aa1-9ea9-cd305ef72622 service cinder] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:05.349959 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9435efd3-4154-4aa1-9ea9-cd305ef72622 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:05.420746 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:05.424311 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9435efd3-4154-4aa1-9ea9-cd305ef72622 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:05.425769 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9435efd3-4154-4aa1-9ea9-cd305ef72622 service cinder] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:05.434713 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1275/2546] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:08:05 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:08:06.068404 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29ea7e8-9fe4-461e-8b90-f1b8299f1421 service cinder] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:06.081128 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:06.131751 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29ea7e8-9fe4-461e-8b90-f1b8299f1421 service cinder] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:06.136454 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcbde08-8b19-4908-9fab-0d27e2a37862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:06.137135 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcbde08-8b19-4908-9fab-0d27e2a37862 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:06.137409 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcbde08-8b19-4908-9fab-0d27e2a37862 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:06.137676 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcbde08-8b19-4908-9fab-0d27e2a37862 admin admin] PATH_INFO: `/v3/users/53ccb1719f4e47e199df29005f34c143` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:06.138835 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcbde08-8b19-4908-9fab-0d27e2a37862 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53ccb1719f4e47e199df29005f34c143)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:06.140804 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcbde08-8b19-4908-9fab-0d27e2a37862 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:06.346932 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-5fcbde08-8b19-4908-9fab-0d27e2a37862 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53ccb1719f4e47e199df29005f34c143'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:06.347594 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-5fcbde08-8b19-4908-9fab-0d27e2a37862 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53ccb1719f4e47e199df29005f34c143'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:06.348189 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-5fcbde08-8b19-4908-9fab-0d27e2a37862 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53ccb1719f4e47e199df29005f34c143'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:06.361148 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-5fcbde08-8b19-4908-9fab-0d27e2a37862 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53ccb1719f4e47e199df29005f34c143'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:06.441522 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-5fcbde08-8b19-4908-9fab-0d27e2a37862 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53ccb1719f4e47e199df29005f34c143'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:06.455117 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1272/2547] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 09:08:06 2022] DELETE /identity/v3/users/53ccb1719f4e47e199df29005f34c143 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:06.459905 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f27ecc-ce47-4042-8f39-c64b1e64a503 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:06.460386 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f27ecc-ce47-4042-8f39-c64b1e64a503 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:06.460732 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f27ecc-ce47-4042-8f39-c64b1e64a503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:06.480978 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-68f27ecc-ce47-4042-8f39-c64b1e64a503 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:06.482198 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-68f27ecc-ce47-4042-8f39-c64b1e64a503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:06.543401 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1276/2548] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:08:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:06.553688 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcbde08-8b19-4908-9fab-0d27e2a37862 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:06.555119 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-5fcbde08-8b19-4908-9fab-0d27e2a37862 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:06.610921 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcbde08-8b19-4908-9fab-0d27e2a37862 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:06.615546 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ca38ca-e64e-40ef-8f8a-337832a8bd82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:06.616177 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ca38ca-e64e-40ef-8f8a-337832a8bd82 service neutron] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:06.616437 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ca38ca-e64e-40ef-8f8a-337832a8bd82 service neutron] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:06.616741 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ca38ca-e64e-40ef-8f8a-337832a8bd82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:06.629526 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ca38ca-e64e-40ef-8f8a-337832a8bd82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:06.631010 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ca38ca-e64e-40ef-8f8a-337832a8bd82 service neutron] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:06.652483 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1273/2549] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:08:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:08:07.055732 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f27ecc-ce47-4042-8f39-c64b1e64a503 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:07.058082 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-68f27ecc-ce47-4042-8f39-c64b1e64a503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:07.079367 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f27ecc-ce47-4042-8f39-c64b1e64a503 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:07.087938 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65966552-2357-47f0-8e5e-c9844564d171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:07.089956 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65966552-2357-47f0-8e5e-c9844564d171 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:07.090650 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65966552-2357-47f0-8e5e-c9844564d171 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:07.091031 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65966552-2357-47f0-8e5e-c9844564d171 admin admin] PATH_INFO: `/v3/projects/e51fee26fe4e46f78e808bd5a0b32006` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:07.092770 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65966552-2357-47f0-8e5e-c9844564d171 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e51fee26fe4e46f78e808bd5a0b32006)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:07.095499 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65966552-2357-47f0-8e5e-c9844564d171 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:07.190527 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-65966552-2357-47f0-8e5e-c9844564d171 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e51fee26fe4e46f78e808bd5a0b32006', 'request_id': 'req-65966552-2357-47f0-8e5e-c9844564d171'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:07.200110 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-65966552-2357-47f0-8e5e-c9844564d171 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e51fee26fe4e46f78e808bd5a0b32006', 'request_id': 'req-65966552-2357-47f0-8e5e-c9844564d171'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:07.276301 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-65966552-2357-47f0-8e5e-c9844564d171 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e51fee26fe4e46f78e808bd5a0b32006', 'request_id': 'req-65966552-2357-47f0-8e5e-c9844564d171'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:07.288892 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-65966552-2357-47f0-8e5e-c9844564d171 admin admin] The token cache is being invalidate because project e51fee26fe4e46f78e808bd5a0b32006 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:08:07.289180 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-65966552-2357-47f0-8e5e-c9844564d171 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:07.289867 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-65966552-2357-47f0-8e5e-c9844564d171 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:07.291350 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1277/2550] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 09:08:07 2022] DELETE /identity/v3/projects/e51fee26fe4e46f78e808bd5a0b32006 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:07.295593 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ca38ca-e64e-40ef-8f8a-337832a8bd82 service neutron] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:07.305821 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ca38ca-e64e-40ef-8f8a-337832a8bd82 service neutron] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:07.339539 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:07.368836 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea8d5fb-0052-4f5e-be8f-472aabafe568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:07.369484 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea8d5fb-0052-4f5e-be8f-472aabafe568 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:07.369796 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea8d5fb-0052-4f5e-be8f-472aabafe568 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:07.370104 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea8d5fb-0052-4f5e-be8f-472aabafe568 admin admin] PATH_INFO: `/v3/users/bd0b1e50bbc04ac28edf3fa1fb3a447a` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:07.371310 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea8d5fb-0052-4f5e-be8f-472aabafe568 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd0b1e50bbc04ac28edf3fa1fb3a447a)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:07.373549 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea8d5fb-0052-4f5e-be8f-472aabafe568 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:07.570069 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-bea8d5fb-0052-4f5e-be8f-472aabafe568 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd0b1e50bbc04ac28edf3fa1fb3a447a'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:07.570788 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-bea8d5fb-0052-4f5e-be8f-472aabafe568 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd0b1e50bbc04ac28edf3fa1fb3a447a'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:07.571489 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-bea8d5fb-0052-4f5e-be8f-472aabafe568 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd0b1e50bbc04ac28edf3fa1fb3a447a'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:07.583438 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-bea8d5fb-0052-4f5e-be8f-472aabafe568 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd0b1e50bbc04ac28edf3fa1fb3a447a'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:07.675937 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-bea8d5fb-0052-4f5e-be8f-472aabafe568 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd0b1e50bbc04ac28edf3fa1fb3a447a'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:07.691414 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1274/2551] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 09:08:07 2022] DELETE /identity/v3/users/bd0b1e50bbc04ac28edf3fa1fb3a447a => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:08:08.415763 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65966552-2357-47f0-8e5e-c9844564d171 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:08.417541 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-65966552-2357-47f0-8e5e-c9844564d171 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:08.430910 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65966552-2357-47f0-8e5e-c9844564d171 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:08.499541 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7ebf3b-cddb-4828-bd69-76f2e2a0d450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:08.500190 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7ebf3b-cddb-4828-bd69-76f2e2a0d450 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:08.500463 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7ebf3b-cddb-4828-bd69-76f2e2a0d450 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:08.500769 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7ebf3b-cddb-4828-bd69-76f2e2a0d450 admin admin] PATH_INFO: `/v3/projects/1f81b11b49184cbdac21b6a3a13ffe65` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:08.501943 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7ebf3b-cddb-4828-bd69-76f2e2a0d450 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f81b11b49184cbdac21b6a3a13ffe65)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:08.503922 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7ebf3b-cddb-4828-bd69-76f2e2a0d450 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:08.527892 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:08.637346 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-8f7ebf3b-cddb-4828-bd69-76f2e2a0d450 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f81b11b49184cbdac21b6a3a13ffe65', 'request_id': 'req-8f7ebf3b-cddb-4828-bd69-76f2e2a0d450'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:08.643074 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-8f7ebf3b-cddb-4828-bd69-76f2e2a0d450 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f81b11b49184cbdac21b6a3a13ffe65', 'request_id': 'req-8f7ebf3b-cddb-4828-bd69-76f2e2a0d450'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:08.715760 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-8f7ebf3b-cddb-4828-bd69-76f2e2a0d450 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f81b11b49184cbdac21b6a3a13ffe65', 'request_id': 'req-8f7ebf3b-cddb-4828-bd69-76f2e2a0d450'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:08.727672 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-8f7ebf3b-cddb-4828-bd69-76f2e2a0d450 admin admin] The token cache is being invalidate because project 1f81b11b49184cbdac21b6a3a13ffe65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:08:08.727958 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-8f7ebf3b-cddb-4828-bd69-76f2e2a0d450 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:08.728642 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-8f7ebf3b-cddb-4828-bd69-76f2e2a0d450 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:08.730126 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1278/2552] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 09:08:08 2022] DELETE /identity/v3/projects/1f81b11b49184cbdac21b6a3a13ffe65 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:08.735092 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8154d8ed-594e-44a9-b8f9-ea8f7e315cbd None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:08.735595 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8154d8ed-594e-44a9-b8f9-ea8f7e315cbd None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:08.735710 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8154d8ed-594e-44a9-b8f9-ea8f7e315cbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:08.758043 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-8154d8ed-594e-44a9-b8f9-ea8f7e315cbd None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:08.759290 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-8154d8ed-594e-44a9-b8f9-ea8f7e315cbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:08.820479 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1275/2553] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:08:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:08.829049 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9bbb5a-676a-454d-a448-1ebed06e4c80 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:08.829331 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9bbb5a-676a-454d-a448-1ebed06e4c80 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:08.829628 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9bbb5a-676a-454d-a448-1ebed06e4c80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:08.849848 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-0c9bbb5a-676a-454d-a448-1ebed06e4c80 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:08.851056 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-0c9bbb5a-676a-454d-a448-1ebed06e4c80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:08.859792 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1279/2554] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:08:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:08.864226 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8154d8ed-594e-44a9-b8f9-ea8f7e315cbd None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:08.874713 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8154d8ed-594e-44a9-b8f9-ea8f7e315cbd None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:08.879464 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a2351c-ec67-4c03-91e0-05a302526a8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:08.880067 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a2351c-ec67-4c03-91e0-05a302526a8b admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:08.880346 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a2351c-ec67-4c03-91e0-05a302526a8b admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:08.881265 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a2351c-ec67-4c03-91e0-05a302526a8b admin admin] PATH_INFO: `/v3/domains` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:08.881910 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a2351c-ec67-4c03-91e0-05a302526a8b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:08.882210 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a2351c-ec67-4c03-91e0-05a302526a8b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:08.883792 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a2351c-ec67-4c03-91e0-05a302526a8b admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:08.895885 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1276/2555] 10.222.0.55 () {58 vars in 1153 bytes} [Wed Aug 24 09:08:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:08.900247 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9bbb5a-676a-454d-a448-1ebed06e4c80 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:08.909752 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9bbb5a-676a-454d-a448-1ebed06e4c80 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:08.914830 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c51858-19a2-4fb6-8495-94680f53e4c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:08.915428 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c51858-19a2-4fb6-8495-94680f53e4c8 admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:08.915702 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c51858-19a2-4fb6-8495-94680f53e4c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:08.915967 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c51858-19a2-4fb6-8495-94680f53e4c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:08.916618 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c51858-19a2-4fb6-8495-94680f53e4c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:08.918522 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c51858-19a2-4fb6-8495-94680f53e4c8 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:08.998576 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1280/2556] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 09:08:08 2022] POST /identity/v3/projects => generated 383 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:08:09.002787 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a2351c-ec67-4c03-91e0-05a302526a8b admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:09.013661 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a2351c-ec67-4c03-91e0-05a302526a8b admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:09.050474 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:09.069097 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5d47e8-31d4-48f3-99d0-6e9b4897cd40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:09.069695 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5d47e8-31d4-48f3-99d0-6e9b4897cd40 admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:09.069972 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5d47e8-31d4-48f3-99d0-6e9b4897cd40 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:09.070243 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5d47e8-31d4-48f3-99d0-6e9b4897cd40 admin admin] PATH_INFO: `/v3/users` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:09.070877 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5d47e8-31d4-48f3-99d0-6e9b4897cd40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:09.072788 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5d47e8-31d4-48f3-99d0-6e9b4897cd40 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:09.168178 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1277/2557] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 09:08:09 2022] POST /identity/v3/users => generated 292 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:08:09.172455 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c51858-19a2-4fb6-8495-94680f53e4c8 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:09.183300 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c51858-19a2-4fb6-8495-94680f53e4c8 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:09.187913 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac678d8a-d3ee-4a42-96a8-930757de26ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:09.188509 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac678d8a-d3ee-4a42-96a8-930757de26ea admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:09.188821 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac678d8a-d3ee-4a42-96a8-930757de26ea admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:09.189118 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac678d8a-d3ee-4a42-96a8-930757de26ea admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:09.189748 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac678d8a-d3ee-4a42-96a8-930757de26ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:09.191323 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac678d8a-d3ee-4a42-96a8-930757de26ea admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:09.202316 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1281/2558] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:08:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:08:09.206480 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5d47e8-31d4-48f3-99d0-6e9b4897cd40 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:09.218034 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5d47e8-31d4-48f3-99d0-6e9b4897cd40 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:09.222789 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c2e4fb-dcad-4bee-8657-0255302e2a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:09.223384 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c2e4fb-dcad-4bee-8657-0255302e2a19 admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:09.223663 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c2e4fb-dcad-4bee-8657-0255302e2a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:09.223937 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c2e4fb-dcad-4bee-8657-0255302e2a19 admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:09.224681 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c2e4fb-dcad-4bee-8657-0255302e2a19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:09.226271 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c2e4fb-dcad-4bee-8657-0255302e2a19 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:09.236742 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1278/2559] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:08:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:08:09.241812 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac678d8a-d3ee-4a42-96a8-930757de26ea admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:09.251922 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac678d8a-d3ee-4a42-96a8-930757de26ea admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:09.257310 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930106f3-8122-4907-b5be-58848b43838e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:09.258207 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930106f3-8122-4907-b5be-58848b43838e admin admin] REQUEST_METHOD: `PUT` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:09.258371 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930106f3-8122-4907-b5be-58848b43838e admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:09.258506 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930106f3-8122-4907-b5be-58848b43838e admin admin] PATH_INFO: `/v3/projects/cfdf8de0f69e4d488f01ea1e041d2144/users/e2bbf3f6ec3f454e8981a0f819990182/roles/b674fd0a592e4e0cada811bba5ceea4d` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:09.273453 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930106f3-8122-4907-b5be-58848b43838e admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfdf8de0f69e4d488f01ea1e041d2144, user_id=e2bbf3f6ec3f454e8981a0f819990182, role_id=b674fd0a592e4e0cada811bba5ceea4d)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:09.276100 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930106f3-8122-4907-b5be-58848b43838e admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:09.368010 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1282/2560] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 09:08:09 2022] PUT /identity/v3/projects/cfdf8de0f69e4d488f01ea1e041d2144/users/e2bbf3f6ec3f454e8981a0f819990182/roles/b674fd0a592e4e0cada811bba5ceea4d => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:09.379562 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab5959c-6e71-46d0-850a-74a88017d904 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:09.381475 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab5959c-6e71-46d0-850a-74a88017d904 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:09.381475 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab5959c-6e71-46d0-850a-74a88017d904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:09.408154 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-fab5959c-6e71-46d0-850a-74a88017d904 None None] MFA Rules not processed for user `e2bbf3f6ec3f454e8981a0f819990182`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:09.409413 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-fab5959c-6e71-46d0-850a-74a88017d904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:09.478148 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1279/2561] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:08:09 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:09.491408 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930106f3-8122-4907-b5be-58848b43838e admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:09.493923 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-930106f3-8122-4907-b5be-58848b43838e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:09.551759 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930106f3-8122-4907-b5be-58848b43838e admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:09.556410 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fd4548-e969-41b7-b02d-1c26a4debb3c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbb9ac667f04c35a63b2689df18efed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:09.557140 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fd4548-e969-41b7-b02d-1c26a4debb3c service cinder] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:09.557419 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fd4548-e969-41b7-b02d-1c26a4debb3c service cinder] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:09.557692 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fd4548-e969-41b7-b02d-1c26a4debb3c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:09.566431 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fd4548-e969-41b7-b02d-1c26a4debb3c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:09.567853 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fd4548-e969-41b7-b02d-1c26a4debb3c service cinder] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:09.577165 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1283/2562] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:08:09 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:08:09.597229 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab5959c-6e71-46d0-850a-74a88017d904 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:09.659990 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab5959c-6e71-46d0-850a-74a88017d904 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:09.665508 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc07611-5588-4699-a5e2-b0f5df313795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:09.666133 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc07611-5588-4699-a5e2-b0f5df313795 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:09.666410 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc07611-5588-4699-a5e2-b0f5df313795 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:09.666683 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc07611-5588-4699-a5e2-b0f5df313795 admin admin] PATH_INFO: `/v3/users/e2bbf3f6ec3f454e8981a0f819990182` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:09.667884 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc07611-5588-4699-a5e2-b0f5df313795 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2bbf3f6ec3f454e8981a0f819990182)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:09.669824 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc07611-5588-4699-a5e2-b0f5df313795 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:09.837905 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-4dc07611-5588-4699-a5e2-b0f5df313795 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2bbf3f6ec3f454e8981a0f819990182'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:09.838610 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-4dc07611-5588-4699-a5e2-b0f5df313795 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2bbf3f6ec3f454e8981a0f819990182'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:09.839191 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-4dc07611-5588-4699-a5e2-b0f5df313795 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2bbf3f6ec3f454e8981a0f819990182'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:09.841093 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:09.852175 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-4dc07611-5588-4699-a5e2-b0f5df313795 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2bbf3f6ec3f454e8981a0f819990182'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:09.949618 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-4dc07611-5588-4699-a5e2-b0f5df313795 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2bbf3f6ec3f454e8981a0f819990182'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:09.963355 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1280/2563] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 09:08:09 2022] DELETE /identity/v3/users/e2bbf3f6ec3f454e8981a0f819990182 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:09.968391 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f176091-4ab9-4db2-8db0-74f4235c5bad None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:09.968695 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f176091-4ab9-4db2-8db0-74f4235c5bad None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:09.968968 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f176091-4ab9-4db2-8db0-74f4235c5bad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:09.988261 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-8f176091-4ab9-4db2-8db0-74f4235c5bad None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:09.989503 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-8f176091-4ab9-4db2-8db0-74f4235c5bad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:10.038362 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:10.056000 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1284/2564] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:08:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:10.065562 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc07611-5588-4699-a5e2-b0f5df313795 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:10.067010 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-4dc07611-5588-4699-a5e2-b0f5df313795 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:10.126796 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc07611-5588-4699-a5e2-b0f5df313795 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:10.131048 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f174d5f3-2005-4128-8f92-25170359fb6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:10.131664 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f174d5f3-2005-4128-8f92-25170359fb6f service neutron] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:10.131942 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f174d5f3-2005-4128-8f92-25170359fb6f service neutron] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:10.132213 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f174d5f3-2005-4128-8f92-25170359fb6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:10.141483 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f174d5f3-2005-4128-8f92-25170359fb6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:10.142893 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f174d5f3-2005-4128-8f92-25170359fb6f service neutron] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:10.152905 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1281/2565] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:08:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:08:10.705854 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f176091-4ab9-4db2-8db0-74f4235c5bad None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:10.707454 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-8f176091-4ab9-4db2-8db0-74f4235c5bad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:10.719717 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f176091-4ab9-4db2-8db0-74f4235c5bad None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:10.724526 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4018028-2d7c-45e1-8b5e-94e64c24a6de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:10.725255 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4018028-2d7c-45e1-8b5e-94e64c24a6de admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:10.725467 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4018028-2d7c-45e1-8b5e-94e64c24a6de admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:10.725782 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4018028-2d7c-45e1-8b5e-94e64c24a6de admin admin] PATH_INFO: `/v3/projects/cfdf8de0f69e4d488f01ea1e041d2144` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:10.727053 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4018028-2d7c-45e1-8b5e-94e64c24a6de admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfdf8de0f69e4d488f01ea1e041d2144)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:10.732575 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4018028-2d7c-45e1-8b5e-94e64c24a6de admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:10.854617 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-b4018028-2d7c-45e1-8b5e-94e64c24a6de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfdf8de0f69e4d488f01ea1e041d2144', 'request_id': 'req-b4018028-2d7c-45e1-8b5e-94e64c24a6de'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:10.860280 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-b4018028-2d7c-45e1-8b5e-94e64c24a6de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfdf8de0f69e4d488f01ea1e041d2144', 'request_id': 'req-b4018028-2d7c-45e1-8b5e-94e64c24a6de'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:10.932912 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-b4018028-2d7c-45e1-8b5e-94e64c24a6de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfdf8de0f69e4d488f01ea1e041d2144', 'request_id': 'req-b4018028-2d7c-45e1-8b5e-94e64c24a6de'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:10.945177 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-b4018028-2d7c-45e1-8b5e-94e64c24a6de admin admin] The token cache is being invalidate because project cfdf8de0f69e4d488f01ea1e041d2144 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:08:10.945494 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-b4018028-2d7c-45e1-8b5e-94e64c24a6de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:10.946121 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-b4018028-2d7c-45e1-8b5e-94e64c24a6de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:10.947628 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1285/2566] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 09:08:10 2022] DELETE /identity/v3/projects/cfdf8de0f69e4d488f01ea1e041d2144 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:10.954017 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96da08a4-151c-40f4-90b5-d2e2393b6f91 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:10.954017 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96da08a4-151c-40f4-90b5-d2e2393b6f91 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:10.954017 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96da08a4-151c-40f4-90b5-d2e2393b6f91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:10.973195 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-96da08a4-151c-40f4-90b5-d2e2393b6f91 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:10.974899 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-96da08a4-151c-40f4-90b5-d2e2393b6f91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:11.033902 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1282/2567] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:08:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:11.042289 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5271d5db-97fd-4e34-aea4-e618805610d3 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:11.042556 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5271d5db-97fd-4e34-aea4-e618805610d3 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:11.042832 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5271d5db-97fd-4e34-aea4-e618805610d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:11.063312 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-5271d5db-97fd-4e34-aea4-e618805610d3 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:11.064515 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-5271d5db-97fd-4e34-aea4-e618805610d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:11.072549 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1286/2568] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:08:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:11.077171 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96da08a4-151c-40f4-90b5-d2e2393b6f91 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:11.086910 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96da08a4-151c-40f4-90b5-d2e2393b6f91 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:11.091810 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa8c559-3227-4e49-b8f5-2760076b4cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:11.092409 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa8c559-3227-4e49-b8f5-2760076b4cfe admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:11.092744 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa8c559-3227-4e49-b8f5-2760076b4cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:11.093018 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa8c559-3227-4e49-b8f5-2760076b4cfe admin admin] PATH_INFO: `/v3/domains` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:11.093651 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa8c559-3227-4e49-b8f5-2760076b4cfe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:11.093955 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa8c559-3227-4e49-b8f5-2760076b4cfe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:11.095537 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa8c559-3227-4e49-b8f5-2760076b4cfe admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:11.107970 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1283/2569] 10.222.0.55 () {58 vars in 1153 bytes} [Wed Aug 24 09:08:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:11.112581 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5271d5db-97fd-4e34-aea4-e618805610d3 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:11.122922 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5271d5db-97fd-4e34-aea4-e618805610d3 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:11.127582 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db6c195-92f0-4df9-b89b-fcf8c5e95b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:11.128197 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db6c195-92f0-4df9-b89b-fcf8c5e95b54 admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:11.128475 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db6c195-92f0-4df9-b89b-fcf8c5e95b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:11.128781 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db6c195-92f0-4df9-b89b-fcf8c5e95b54 admin admin] PATH_INFO: `/v3/projects` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:11.129396 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db6c195-92f0-4df9-b89b-fcf8c5e95b54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:11.131354 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db6c195-92f0-4df9-b89b-fcf8c5e95b54 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:11.236993 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1287/2570] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 09:08:11 2022] POST /identity/v3/projects => generated 377 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:08:11.241586 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa8c559-3227-4e49-b8f5-2760076b4cfe admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:11.252256 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa8c559-3227-4e49-b8f5-2760076b4cfe admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:11.311045 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:11.317577 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcc63b6-279f-4445-b484-0dd22654966f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:11.318193 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcc63b6-279f-4445-b484-0dd22654966f admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:11.318477 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcc63b6-279f-4445-b484-0dd22654966f admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:11.318750 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcc63b6-279f-4445-b484-0dd22654966f admin admin] PATH_INFO: `/v3/users` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:11.319383 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcc63b6-279f-4445-b484-0dd22654966f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:11.321349 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcc63b6-279f-4445-b484-0dd22654966f admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:11.441709 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1284/2571] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 09:08:11 2022] POST /identity/v3/users => generated 289 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:08:11.447004 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db6c195-92f0-4df9-b89b-fcf8c5e95b54 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:11.460456 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db6c195-92f0-4df9-b89b-fcf8c5e95b54 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:11.465747 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1161178-8c97-402f-8d37-b8beafd3d85a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:11.466360 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1161178-8c97-402f-8d37-b8beafd3d85a admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:11.466637 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1161178-8c97-402f-8d37-b8beafd3d85a admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:11.466906 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1161178-8c97-402f-8d37-b8beafd3d85a admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:11.467547 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1161178-8c97-402f-8d37-b8beafd3d85a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:11.469174 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1161178-8c97-402f-8d37-b8beafd3d85a admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:11.483931 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1288/2572] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:08:11 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:08:11.485410 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcc63b6-279f-4445-b484-0dd22654966f admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:11.495888 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcc63b6-279f-4445-b484-0dd22654966f admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:11.500742 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bc09bc-1060-4ab2-914b-d86077b4ae47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:11.501520 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bc09bc-1060-4ab2-914b-d86077b4ae47 admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:11.501919 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bc09bc-1060-4ab2-914b-d86077b4ae47 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:11.502205 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bc09bc-1060-4ab2-914b-d86077b4ae47 admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:11.502992 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bc09bc-1060-4ab2-914b-d86077b4ae47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:11.504756 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bc09bc-1060-4ab2-914b-d86077b4ae47 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:11.517947 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1285/2573] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:08:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:08:11.529977 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1161178-8c97-402f-8d37-b8beafd3d85a admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:11.540627 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1161178-8c97-402f-8d37-b8beafd3d85a admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:11.545522 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df7a8f7-4c25-49df-82b5-318aca8c1d5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:11.546208 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df7a8f7-4c25-49df-82b5-318aca8c1d5b admin admin] REQUEST_METHOD: `PUT` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:11.546485 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df7a8f7-4c25-49df-82b5-318aca8c1d5b admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:11.546764 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df7a8f7-4c25-49df-82b5-318aca8c1d5b admin admin] PATH_INFO: `/v3/projects/867ee5daa3104a42af238b3ff851123f/users/a85aef96518f42769a3873d7f6bf4a43/roles/b674fd0a592e4e0cada811bba5ceea4d` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:11.563075 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df7a8f7-4c25-49df-82b5-318aca8c1d5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=867ee5daa3104a42af238b3ff851123f, user_id=a85aef96518f42769a3873d7f6bf4a43, role_id=b674fd0a592e4e0cada811bba5ceea4d)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:11.584222 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df7a8f7-4c25-49df-82b5-318aca8c1d5b admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:11.722392 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1289/2574] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 09:08:11 2022] PUT /identity/v3/projects/867ee5daa3104a42af238b3ff851123f/users/a85aef96518f42769a3873d7f6bf4a43/roles/b674fd0a592e4e0cada811bba5ceea4d => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:11.732745 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a215246-680c-47a2-ad86-3dc0c1402011 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:11.733031 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a215246-680c-47a2-ad86-3dc0c1402011 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:11.733351 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a215246-680c-47a2-ad86-3dc0c1402011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:11.884808 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-5a215246-680c-47a2-ad86-3dc0c1402011 None None] MFA Rules not processed for user `a85aef96518f42769a3873d7f6bf4a43`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:11.886064 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-5a215246-680c-47a2-ad86-3dc0c1402011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:11.950830 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1286/2575] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:08:11 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:11.962332 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df7a8f7-4c25-49df-82b5-318aca8c1d5b admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:11.965147 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-1df7a8f7-4c25-49df-82b5-318aca8c1d5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:12.025032 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df7a8f7-4c25-49df-82b5-318aca8c1d5b admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:12.028962 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b1fb50-8a80-4e30-b2f1-d96b2db6576b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbb9ac667f04c35a63b2689df18efed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:12.033980 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b1fb50-8a80-4e30-b2f1-d96b2db6576b service cinder] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:12.033980 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b1fb50-8a80-4e30-b2f1-d96b2db6576b service cinder] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:12.033980 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b1fb50-8a80-4e30-b2f1-d96b2db6576b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:12.038643 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b1fb50-8a80-4e30-b2f1-d96b2db6576b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:12.040076 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b1fb50-8a80-4e30-b2f1-d96b2db6576b service cinder] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:12.048974 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1290/2576] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:08:11 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:08:21.947890 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=40,keystone:INSERT=3 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:22.044895 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=86,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:44.513523 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a215246-680c-47a2-ad86-3dc0c1402011 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:44.576624 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a215246-680c-47a2-ad86-3dc0c1402011 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:44.580992 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9f012c-1301-4d44-9782-2f5c1c174486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:44.581621 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9f012c-1301-4d44-9782-2f5c1c174486 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:44.581899 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9f012c-1301-4d44-9782-2f5c1c174486 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:44.582171 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9f012c-1301-4d44-9782-2f5c1c174486 admin admin] PATH_INFO: `/v3/users/a85aef96518f42769a3873d7f6bf4a43` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:44.583292 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9f012c-1301-4d44-9782-2f5c1c174486 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a85aef96518f42769a3873d7f6bf4a43)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:44.585239 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9f012c-1301-4d44-9782-2f5c1c174486 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:44.766748 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-fa9f012c-1301-4d44-9782-2f5c1c174486 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a85aef96518f42769a3873d7f6bf4a43'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:44.767511 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-fa9f012c-1301-4d44-9782-2f5c1c174486 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a85aef96518f42769a3873d7f6bf4a43'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:44.768124 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-fa9f012c-1301-4d44-9782-2f5c1c174486 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a85aef96518f42769a3873d7f6bf4a43'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:44.780990 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-fa9f012c-1301-4d44-9782-2f5c1c174486 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a85aef96518f42769a3873d7f6bf4a43'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:44.858343 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-fa9f012c-1301-4d44-9782-2f5c1c174486 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a85aef96518f42769a3873d7f6bf4a43'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:44.872263 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1287/2577] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 09:08:44 2022] DELETE /identity/v3/users/a85aef96518f42769a3873d7f6bf4a43 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:44.877351 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94651707-5ff8-4b4c-a3a0-cb63c9cf7bce None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:44.877896 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94651707-5ff8-4b4c-a3a0-cb63c9cf7bce None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:44.877896 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94651707-5ff8-4b4c-a3a0-cb63c9cf7bce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:44.898625 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-94651707-5ff8-4b4c-a3a0-cb63c9cf7bce None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:44.899903 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-94651707-5ff8-4b4c-a3a0-cb63c9cf7bce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:44.966108 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1291/2578] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:08:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:44.976528 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9f012c-1301-4d44-9782-2f5c1c174486 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:44.977978 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-fa9f012c-1301-4d44-9782-2f5c1c174486 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:45.038379 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9f012c-1301-4d44-9782-2f5c1c174486 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:45.042646 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70284861-4084-489f-9e48-285c0e19e2a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:45.047107 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70284861-4084-489f-9e48-285c0e19e2a8 service neutron] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:45.047107 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70284861-4084-489f-9e48-285c0e19e2a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:45.047107 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70284861-4084-489f-9e48-285c0e19e2a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:45.053545 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70284861-4084-489f-9e48-285c0e19e2a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:45.054990 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70284861-4084-489f-9e48-285c0e19e2a8 service neutron] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:45.065258 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1288/2579] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:08:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:08:45.467421 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94651707-5ff8-4b4c-a3a0-cb63c9cf7bce None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:45.467421 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-94651707-5ff8-4b4c-a3a0-cb63c9cf7bce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:45.480847 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94651707-5ff8-4b4c-a3a0-cb63c9cf7bce None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:45.485783 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f61192-8894-4e17-9f8d-b695dd26f550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:45.486465 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f61192-8894-4e17-9f8d-b695dd26f550 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:45.486741 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f61192-8894-4e17-9f8d-b695dd26f550 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:45.487013 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f61192-8894-4e17-9f8d-b695dd26f550 admin admin] PATH_INFO: `/v3/projects/867ee5daa3104a42af238b3ff851123f` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:45.488157 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f61192-8894-4e17-9f8d-b695dd26f550 admin admin] RBAC: Authorizing `identity:delete_project(project_id=867ee5daa3104a42af238b3ff851123f)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:45.492405 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f61192-8894-4e17-9f8d-b695dd26f550 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:45.609251 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-f2f61192-8894-4e17-9f8d-b695dd26f550 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '867ee5daa3104a42af238b3ff851123f', 'request_id': 'req-f2f61192-8894-4e17-9f8d-b695dd26f550'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:45.615186 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-f2f61192-8894-4e17-9f8d-b695dd26f550 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '867ee5daa3104a42af238b3ff851123f', 'request_id': 'req-f2f61192-8894-4e17-9f8d-b695dd26f550'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:45.687056 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-f2f61192-8894-4e17-9f8d-b695dd26f550 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '867ee5daa3104a42af238b3ff851123f', 'request_id': 'req-f2f61192-8894-4e17-9f8d-b695dd26f550'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:45.699314 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-f2f61192-8894-4e17-9f8d-b695dd26f550 admin admin] The token cache is being invalidate because project 867ee5daa3104a42af238b3ff851123f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:08:45.699604 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-f2f61192-8894-4e17-9f8d-b695dd26f550 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:45.700257 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-f2f61192-8894-4e17-9f8d-b695dd26f550 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:45.701773 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1292/2580] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 09:08:45 2022] DELETE /identity/v3/projects/867ee5daa3104a42af238b3ff851123f => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:45.707728 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70e217e-bec9-4b51-be8c-09007df9732d None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:45.708241 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70e217e-bec9-4b51-be8c-09007df9732d None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:45.708400 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70e217e-bec9-4b51-be8c-09007df9732d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:45.729537 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-d70e217e-bec9-4b51-be8c-09007df9732d None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:45.730797 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-d70e217e-bec9-4b51-be8c-09007df9732d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:45.750843 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:45.797254 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1289/2581] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:08:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:45.805782 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b5ed78-883d-4bf4-b2d6-61a76c891880 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:45.806068 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b5ed78-883d-4bf4-b2d6-61a76c891880 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:45.806393 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b5ed78-883d-4bf4-b2d6-61a76c891880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:45.830744 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-91b5ed78-883d-4bf4-b2d6-61a76c891880 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:45.831962 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-91b5ed78-883d-4bf4-b2d6-61a76c891880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:45.841590 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1293/2582] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:08:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:45.845893 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70e217e-bec9-4b51-be8c-09007df9732d None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:45.856874 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70e217e-bec9-4b51-be8c-09007df9732d None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:45.861754 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fc690a-9588-4475-b2f7-323014870a86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:45.862564 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fc690a-9588-4475-b2f7-323014870a86 admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:45.865772 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fc690a-9588-4475-b2f7-323014870a86 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:45.865772 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fc690a-9588-4475-b2f7-323014870a86 admin admin] PATH_INFO: `/v3/domains` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:45.865772 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fc690a-9588-4475-b2f7-323014870a86 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:45.865772 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fc690a-9588-4475-b2f7-323014870a86 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:45.865772 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fc690a-9588-4475-b2f7-323014870a86 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:45.881108 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1290/2583] 10.222.0.55 () {58 vars in 1153 bytes} [Wed Aug 24 09:08:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:45.885427 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b5ed78-883d-4bf4-b2d6-61a76c891880 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:45.895436 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b5ed78-883d-4bf4-b2d6-61a76c891880 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:45.900825 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d59b78-cdbf-447a-81b1-21a1b7bd12ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:45.901452 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d59b78-cdbf-447a-81b1-21a1b7bd12ca admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:45.901733 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d59b78-cdbf-447a-81b1-21a1b7bd12ca admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:45.902008 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d59b78-cdbf-447a-81b1-21a1b7bd12ca admin admin] PATH_INFO: `/v3/projects` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:45.902652 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d59b78-cdbf-447a-81b1-21a1b7bd12ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:45.904605 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d59b78-cdbf-447a-81b1-21a1b7bd12ca admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:45.992642 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1294/2584] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 09:08:45 2022] POST /identity/v3/projects => generated 373 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:08:45.994310 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fc690a-9588-4475-b2f7-323014870a86 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:46.009351 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fc690a-9588-4475-b2f7-323014870a86 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:46.077121 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b83bf7-11ff-4944-b039-90e4c10252cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:46.077121 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b83bf7-11ff-4944-b039-90e4c10252cc admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:46.077121 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b83bf7-11ff-4944-b039-90e4c10252cc admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:46.077121 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b83bf7-11ff-4944-b039-90e4c10252cc admin admin] PATH_INFO: `/v3/users` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:46.079008 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b83bf7-11ff-4944-b039-90e4c10252cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:46.080794 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b83bf7-11ff-4944-b039-90e4c10252cc admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:46.161279 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1291/2585] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 09:08:45 2022] POST /identity/v3/users => generated 287 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:08:46.165154 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d59b78-cdbf-447a-81b1-21a1b7bd12ca admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:46.175677 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d59b78-cdbf-447a-81b1-21a1b7bd12ca admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:46.180450 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f679647f-a245-4db6-9663-4d8f21a15a7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:46.181132 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f679647f-a245-4db6-9663-4d8f21a15a7a admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:46.181369 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f679647f-a245-4db6-9663-4d8f21a15a7a admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:46.181635 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f679647f-a245-4db6-9663-4d8f21a15a7a admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:46.182273 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f679647f-a245-4db6-9663-4d8f21a15a7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:46.183912 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f679647f-a245-4db6-9663-4d8f21a15a7a admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:46.197258 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1295/2586] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:08:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:08:46.199434 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b83bf7-11ff-4944-b039-90e4c10252cc admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:46.210035 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b83bf7-11ff-4944-b039-90e4c10252cc admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:46.215477 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783df9ba-495e-4df8-8c01-3689f18d0dd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:46.216078 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783df9ba-495e-4df8-8c01-3689f18d0dd0 admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:46.216357 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783df9ba-495e-4df8-8c01-3689f18d0dd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:46.216662 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783df9ba-495e-4df8-8c01-3689f18d0dd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:46.217395 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783df9ba-495e-4df8-8c01-3689f18d0dd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:46.218999 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783df9ba-495e-4df8-8c01-3689f18d0dd0 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:46.230214 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1292/2587] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:08:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:08:46.234503 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f679647f-a245-4db6-9663-4d8f21a15a7a admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:46.245768 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f679647f-a245-4db6-9663-4d8f21a15a7a admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:46.250851 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbde594-cd2b-4931-9690-9a75a662e7e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:46.251523 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbde594-cd2b-4931-9690-9a75a662e7e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:46.251806 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbde594-cd2b-4931-9690-9a75a662e7e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:46.252091 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbde594-cd2b-4931-9690-9a75a662e7e1 admin admin] PATH_INFO: `/v3/projects/bccb7b3c5c2844eb84a115fa60b186ff/users/5c030e83e1cc40609d901de3ed82a90e/roles/b674fd0a592e4e0cada811bba5ceea4d` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:46.269554 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbde594-cd2b-4931-9690-9a75a662e7e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bccb7b3c5c2844eb84a115fa60b186ff, user_id=5c030e83e1cc40609d901de3ed82a90e, role_id=b674fd0a592e4e0cada811bba5ceea4d)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:46.272223 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbde594-cd2b-4931-9690-9a75a662e7e1 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:46.332891 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1296/2588] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 09:08:46 2022] PUT /identity/v3/projects/bccb7b3c5c2844eb84a115fa60b186ff/users/5c030e83e1cc40609d901de3ed82a90e/roles/b674fd0a592e4e0cada811bba5ceea4d => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:46.355589 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5f00c3-14e4-4d61-b967-c238c09790aa None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:46.355936 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5f00c3-14e4-4d61-b967-c238c09790aa None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:46.356274 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5f00c3-14e4-4d61-b967-c238c09790aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:46.387551 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-ef5f00c3-14e4-4d61-b967-c238c09790aa None None] MFA Rules not processed for user `5c030e83e1cc40609d901de3ed82a90e`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:46.388830 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-ef5f00c3-14e4-4d61-b967-c238c09790aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:46.458198 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1293/2589] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:08:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:46.466428 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbde594-cd2b-4931-9690-9a75a662e7e1 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:46.468605 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-2cbde594-cd2b-4931-9690-9a75a662e7e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:46.492264 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:46.530401 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbde594-cd2b-4931-9690-9a75a662e7e1 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:46.534096 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d7ee6c-0e9d-45d3-8423-a19b468a6d31 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbb9ac667f04c35a63b2689df18efed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:46.534749 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d7ee6c-0e9d-45d3-8423-a19b468a6d31 service cinder] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:46.535030 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d7ee6c-0e9d-45d3-8423-a19b468a6d31 service cinder] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:46.535301 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d7ee6c-0e9d-45d3-8423-a19b468a6d31 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:46.544914 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d7ee6c-0e9d-45d3-8423-a19b468a6d31 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:46.546346 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d7ee6c-0e9d-45d3-8423-a19b468a6d31 service cinder] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:46.555680 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1297/2590] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:08:46 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:08:47.489759 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5f00c3-14e4-4d61-b967-c238c09790aa None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:47.491182 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-ef5f00c3-14e4-4d61-b967-c238c09790aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:47.513218 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:47.557346 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5f00c3-14e4-4d61-b967-c238c09790aa None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:47.561523 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8097e3-7ed4-4b8f-bfe9-8b1deebb8b24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:47.562344 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8097e3-7ed4-4b8f-bfe9-8b1deebb8b24 service neutron] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:47.562810 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8097e3-7ed4-4b8f-bfe9-8b1deebb8b24 service neutron] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:47.563173 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8097e3-7ed4-4b8f-bfe9-8b1deebb8b24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:47.565140 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-7f8097e3-7ed4-4b8f-bfe9-8b1deebb8b24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:47.631781 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8097e3-7ed4-4b8f-bfe9-8b1deebb8b24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:47.634075 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8097e3-7ed4-4b8f-bfe9-8b1deebb8b24 service neutron] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:47.644710 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1294/2591] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:08:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:08:49.519263 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f308dfb-edab-4288-8f86-57a97f9dd237 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:49.519847 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f308dfb-edab-4288-8f86-57a97f9dd237 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:49.519847 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f308dfb-edab-4288-8f86-57a97f9dd237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:49.540603 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-5f308dfb-edab-4288-8f86-57a97f9dd237 None None] MFA Rules not processed for user `5c030e83e1cc40609d901de3ed82a90e`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:49.541850 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-5f308dfb-edab-4288-8f86-57a97f9dd237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:49.549272 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1298/2592] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:08:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:51.620288 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8097e3-7ed4-4b8f-bfe9-8b1deebb8b24 service neutron] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:51.621914 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-7f8097e3-7ed4-4b8f-bfe9-8b1deebb8b24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:51.638185 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8097e3-7ed4-4b8f-bfe9-8b1deebb8b24 service neutron] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:51.643411 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1378ba76-22a6-4561-9766-a3bef7510972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:51.644125 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1378ba76-22a6-4561-9766-a3bef7510972 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:51.644404 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1378ba76-22a6-4561-9766-a3bef7510972 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:51.644775 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1378ba76-22a6-4561-9766-a3bef7510972 admin admin] PATH_INFO: `/v3/users/993697d33b7841668b1eabcdcd6b40f9` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:51.645864 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1378ba76-22a6-4561-9766-a3bef7510972 admin admin] RBAC: Authorizing `identity:delete_user(user_id=993697d33b7841668b1eabcdcd6b40f9)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:51.647773 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1378ba76-22a6-4561-9766-a3bef7510972 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:51.849576 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-1378ba76-22a6-4561-9766-a3bef7510972 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '993697d33b7841668b1eabcdcd6b40f9'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:51.850257 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-1378ba76-22a6-4561-9766-a3bef7510972 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '993697d33b7841668b1eabcdcd6b40f9'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:51.851009 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-1378ba76-22a6-4561-9766-a3bef7510972 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '993697d33b7841668b1eabcdcd6b40f9'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:51.863498 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-1378ba76-22a6-4561-9766-a3bef7510972 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '993697d33b7841668b1eabcdcd6b40f9'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:51.955166 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-1378ba76-22a6-4561-9766-a3bef7510972 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '993697d33b7841668b1eabcdcd6b40f9'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:51.969196 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1295/2593] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 09:08:51 2022] DELETE /identity/v3/users/993697d33b7841668b1eabcdcd6b40f9 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:52.282977 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f308dfb-edab-4288-8f86-57a97f9dd237 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:52.296393 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f308dfb-edab-4288-8f86-57a97f9dd237 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:52.355586 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc031691-f879-427b-a30c-6b14f0af45ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:52.356313 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc031691-f879-427b-a30c-6b14f0af45ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:52.356695 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc031691-f879-427b-a30c-6b14f0af45ee admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:52.357037 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc031691-f879-427b-a30c-6b14f0af45ee admin admin] PATH_INFO: `/v3/projects/3809ac47fe214dda92729d4c89648a17` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:52.358233 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc031691-f879-427b-a30c-6b14f0af45ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=3809ac47fe214dda92729d4c89648a17)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:52.360243 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc031691-f879-427b-a30c-6b14f0af45ee admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:52.464328 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-bc031691-f879-427b-a30c-6b14f0af45ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3809ac47fe214dda92729d4c89648a17', 'request_id': 'req-bc031691-f879-427b-a30c-6b14f0af45ee'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:52.470109 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-bc031691-f879-427b-a30c-6b14f0af45ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3809ac47fe214dda92729d4c89648a17', 'request_id': 'req-bc031691-f879-427b-a30c-6b14f0af45ee'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:52.527242 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-bc031691-f879-427b-a30c-6b14f0af45ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3809ac47fe214dda92729d4c89648a17', 'request_id': 'req-bc031691-f879-427b-a30c-6b14f0af45ee'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:52.539173 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-bc031691-f879-427b-a30c-6b14f0af45ee admin admin] The token cache is being invalidate because project 3809ac47fe214dda92729d4c89648a17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:08:52.539456 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-bc031691-f879-427b-a30c-6b14f0af45ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:52.540102 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-bc031691-f879-427b-a30c-6b14f0af45ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:52.541641 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1299/2594] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 09:08:52 2022] DELETE /identity/v3/projects/3809ac47fe214dda92729d4c89648a17 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:52.547005 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0389c9a6-3cd7-4c0e-bb82-bb09802e3877 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:52.547256 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0389c9a6-3cd7-4c0e-bb82-bb09802e3877 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:52.547525 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0389c9a6-3cd7-4c0e-bb82-bb09802e3877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:52.566686 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-0389c9a6-3cd7-4c0e-bb82-bb09802e3877 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:52.567875 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-0389c9a6-3cd7-4c0e-bb82-bb09802e3877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:52.602666 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:52.629162 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1296/2595] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:08:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:52.637451 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6b9fb8-7652-4154-935b-60ba454dcb03 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:52.637736 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6b9fb8-7652-4154-935b-60ba454dcb03 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:52.638013 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6b9fb8-7652-4154-935b-60ba454dcb03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:52.658711 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-cb6b9fb8-7652-4154-935b-60ba454dcb03 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:52.659893 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-cb6b9fb8-7652-4154-935b-60ba454dcb03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:52.668645 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1300/2596] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:08:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:52.672889 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0389c9a6-3cd7-4c0e-bb82-bb09802e3877 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:52.682522 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0389c9a6-3cd7-4c0e-bb82-bb09802e3877 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:52.687823 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a82be4-68f8-41fa-b392-a72226fe7199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:52.688429 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a82be4-68f8-41fa-b392-a72226fe7199 admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:52.688733 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a82be4-68f8-41fa-b392-a72226fe7199 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:52.689007 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a82be4-68f8-41fa-b392-a72226fe7199 admin admin] PATH_INFO: `/v3/domains` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:52.689649 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a82be4-68f8-41fa-b392-a72226fe7199 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:52.689956 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a82be4-68f8-41fa-b392-a72226fe7199 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:52.691660 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a82be4-68f8-41fa-b392-a72226fe7199 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:52.703693 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1297/2597] 10.222.0.55 () {58 vars in 1153 bytes} [Wed Aug 24 09:08:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:52.707921 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6b9fb8-7652-4154-935b-60ba454dcb03 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:52.717918 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6b9fb8-7652-4154-935b-60ba454dcb03 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:52.722724 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303db66d-8453-4ae3-81bf-6f27cd51ae47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:52.723324 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303db66d-8453-4ae3-81bf-6f27cd51ae47 admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:52.723602 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303db66d-8453-4ae3-81bf-6f27cd51ae47 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:52.723878 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303db66d-8453-4ae3-81bf-6f27cd51ae47 admin admin] PATH_INFO: `/v3/projects` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:52.724510 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303db66d-8453-4ae3-81bf-6f27cd51ae47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:52.726449 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303db66d-8453-4ae3-81bf-6f27cd51ae47 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:52.808000 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1301/2598] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 09:08:52 2022] POST /identity/v3/projects => generated 385 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:08:52.812423 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a82be4-68f8-41fa-b392-a72226fe7199 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:52.822569 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a82be4-68f8-41fa-b392-a72226fe7199 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:52.879320 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fb5245-c64a-4d33-8095-a9955b7319f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:52.879884 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb5245-c64a-4d33-8095-a9955b7319f4 admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:52.880167 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb5245-c64a-4d33-8095-a9955b7319f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:52.880442 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb5245-c64a-4d33-8095-a9955b7319f4 admin admin] PATH_INFO: `/v3/users` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:52.881096 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fb5245-c64a-4d33-8095-a9955b7319f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:52.882997 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fb5245-c64a-4d33-8095-a9955b7319f4 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:52.972692 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1298/2599] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 09:08:52 2022] POST /identity/v3/users => generated 293 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:08:52.976999 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303db66d-8453-4ae3-81bf-6f27cd51ae47 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:52.988485 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303db66d-8453-4ae3-81bf-6f27cd51ae47 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:52.994039 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ab5d76-ef42-4fec-ba4b-7a62dbb274b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:52.994039 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ab5d76-ef42-4fec-ba4b-7a62dbb274b2 admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:52.994039 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ab5d76-ef42-4fec-ba4b-7a62dbb274b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:52.994372 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ab5d76-ef42-4fec-ba4b-7a62dbb274b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:52.995502 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ab5d76-ef42-4fec-ba4b-7a62dbb274b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:52.997114 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ab5d76-ef42-4fec-ba4b-7a62dbb274b2 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:53.007071 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1302/2600] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:08:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:08:53.011710 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fb5245-c64a-4d33-8095-a9955b7319f4 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:53.021799 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fb5245-c64a-4d33-8095-a9955b7319f4 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:53.026447 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6bf0a8-5050-4af5-b150-9e0f062954c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:53.027036 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6bf0a8-5050-4af5-b150-9e0f062954c3 admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:53.027315 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6bf0a8-5050-4af5-b150-9e0f062954c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:53.027589 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6bf0a8-5050-4af5-b150-9e0f062954c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:53.028231 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6bf0a8-5050-4af5-b150-9e0f062954c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:53.029972 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6bf0a8-5050-4af5-b150-9e0f062954c3 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:53.040641 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1299/2601] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:08:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:08:53.045101 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ab5d76-ef42-4fec-ba4b-7a62dbb274b2 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:53.054952 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ab5d76-ef42-4fec-ba4b-7a62dbb274b2 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:53.060469 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae25e2c-be24-48f3-ba4f-9083ff73b661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:53.061220 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae25e2c-be24-48f3-ba4f-9083ff73b661 admin admin] REQUEST_METHOD: `PUT` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:53.061528 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae25e2c-be24-48f3-ba4f-9083ff73b661 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:53.061804 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae25e2c-be24-48f3-ba4f-9083ff73b661 admin admin] PATH_INFO: `/v3/projects/02ed9f0eac5849358cf11723e1a51fee/users/d6bab96f2a6546d9903cf4b57b90780c/roles/b674fd0a592e4e0cada811bba5ceea4d` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:53.064090 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:53.077608 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae25e2c-be24-48f3-ba4f-9083ff73b661 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02ed9f0eac5849358cf11723e1a51fee, user_id=d6bab96f2a6546d9903cf4b57b90780c, role_id=b674fd0a592e4e0cada811bba5ceea4d)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:53.080270 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae25e2c-be24-48f3-ba4f-9083ff73b661 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:53.149873 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1303/2602] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 09:08:53 2022] PUT /identity/v3/projects/02ed9f0eac5849358cf11723e1a51fee/users/d6bab96f2a6546d9903cf4b57b90780c/roles/b674fd0a592e4e0cada811bba5ceea4d => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:53.161540 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4153a70f-c11f-4b55-9080-1b9311733fbd None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:53.161821 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4153a70f-c11f-4b55-9080-1b9311733fbd None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:53.162096 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4153a70f-c11f-4b55-9080-1b9311733fbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:53.191283 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-4153a70f-c11f-4b55-9080-1b9311733fbd None None] MFA Rules not processed for user `d6bab96f2a6546d9903cf4b57b90780c`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:53.192597 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-4153a70f-c11f-4b55-9080-1b9311733fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:53.261603 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1300/2603] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:08:53 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:08:53.272848 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae25e2c-be24-48f3-ba4f-9083ff73b661 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:53.274718 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-0ae25e2c-be24-48f3-ba4f-9083ff73b661 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:53.360167 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae25e2c-be24-48f3-ba4f-9083ff73b661 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:53.365993 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ee94a6-2b4f-4f04-ab11-3014de564849 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17da357e555420799ee681bffbcc268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:53.366608 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ee94a6-2b4f-4f04-ab11-3014de564849 service nova] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:53.366891 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ee94a6-2b4f-4f04-ab11-3014de564849 service nova] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:53.367163 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ee94a6-2b4f-4f04-ab11-3014de564849 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:53.375953 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ee94a6-2b4f-4f04-ab11-3014de564849 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:53.377572 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ee94a6-2b4f-4f04-ab11-3014de564849 service nova] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:53.387309 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1304/2604] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:08:53 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:09:03.257879 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=81,keystone:INSERT=3 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:03.382618 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=41,keystone:INSERT=1 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:04.732498 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4153a70f-c11f-4b55-9080-1b9311733fbd None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:04.732498 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-4153a70f-c11f-4b55-9080-1b9311733fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:04.796511 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4153a70f-c11f-4b55-9080-1b9311733fbd None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:04.800444 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68e6d96-7542-4561-b07f-2678606199f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:04.801093 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68e6d96-7542-4561-b07f-2678606199f3 service neutron] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:04.801610 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68e6d96-7542-4561-b07f-2678606199f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:04.801610 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68e6d96-7542-4561-b07f-2678606199f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:04.803413 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-a68e6d96-7542-4561-b07f-2678606199f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:04.870866 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68e6d96-7542-4561-b07f-2678606199f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:04.872301 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68e6d96-7542-4561-b07f-2678606199f3 service neutron] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:04.882034 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1301/2605] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:09:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 09:09:05.956089 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ee94a6-2b4f-4f04-ab11-3014de564849 service nova] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:05.964628 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ee94a6-2b4f-4f04-ab11-3014de564849 service nova] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:05.970372 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c92ca6d-44a0-4bfd-9adf-f3766be093f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:05.971089 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c92ca6d-44a0-4bfd-9adf-f3766be093f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:05.971452 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c92ca6d-44a0-4bfd-9adf-f3766be093f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:05.971801 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c92ca6d-44a0-4bfd-9adf-f3766be093f6 admin admin] PATH_INFO: `/v3/users/d6bab96f2a6546d9903cf4b57b90780c` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:05.973115 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c92ca6d-44a0-4bfd-9adf-f3766be093f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6bab96f2a6546d9903cf4b57b90780c)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:05.975228 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c92ca6d-44a0-4bfd-9adf-f3766be093f6 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:06.183377 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-8c92ca6d-44a0-4bfd-9adf-f3766be093f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6bab96f2a6546d9903cf4b57b90780c'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:06.184037 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-8c92ca6d-44a0-4bfd-9adf-f3766be093f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6bab96f2a6546d9903cf4b57b90780c'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:06.184688 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-8c92ca6d-44a0-4bfd-9adf-f3766be093f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6bab96f2a6546d9903cf4b57b90780c'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:06.201687 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-8c92ca6d-44a0-4bfd-9adf-f3766be093f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6bab96f2a6546d9903cf4b57b90780c'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:06.252626 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-8c92ca6d-44a0-4bfd-9adf-f3766be093f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6bab96f2a6546d9903cf4b57b90780c'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:06.269469 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1305/2606] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 09:09:05 2022] DELETE /identity/v3/users/d6bab96f2a6546d9903cf4b57b90780c => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:09:06.274167 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9651bf69-25d5-4bb4-b2b4-ba3c52dcd8b1 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:06.274666 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9651bf69-25d5-4bb4-b2b4-ba3c52dcd8b1 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:06.274783 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9651bf69-25d5-4bb4-b2b4-ba3c52dcd8b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:06.294796 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-9651bf69-25d5-4bb4-b2b4-ba3c52dcd8b1 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:09:06.296015 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-9651bf69-25d5-4bb4-b2b4-ba3c52dcd8b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:06.360955 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1302/2607] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:09:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:09:06.371095 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c92ca6d-44a0-4bfd-9adf-f3766be093f6 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:06.372648 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-8c92ca6d-44a0-4bfd-9adf-f3766be093f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:06.437808 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c92ca6d-44a0-4bfd-9adf-f3766be093f6 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:06.441462 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9045690e-f934-4e5c-8fee-1e165dc034c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:06.442076 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9045690e-f934-4e5c-8fee-1e165dc034c2 service neutron] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:06.442353 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9045690e-f934-4e5c-8fee-1e165dc034c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:06.442625 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9045690e-f934-4e5c-8fee-1e165dc034c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:06.453261 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9045690e-f934-4e5c-8fee-1e165dc034c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:06.454720 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9045690e-f934-4e5c-8fee-1e165dc034c2 service neutron] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:06.466910 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1306/2608] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:09:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:09:06.928883 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9651bf69-25d5-4bb4-b2b4-ba3c52dcd8b1 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:06.935352 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-9651bf69-25d5-4bb4-b2b4-ba3c52dcd8b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:06.951998 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9651bf69-25d5-4bb4-b2b4-ba3c52dcd8b1 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:06.957364 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93025130-bab8-4d4f-92f6-fd8ebb4bf73c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:06.958142 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93025130-bab8-4d4f-92f6-fd8ebb4bf73c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:06.958512 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93025130-bab8-4d4f-92f6-fd8ebb4bf73c admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:06.959063 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93025130-bab8-4d4f-92f6-fd8ebb4bf73c admin admin] PATH_INFO: `/v3/projects/02ed9f0eac5849358cf11723e1a51fee` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:06.960743 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93025130-bab8-4d4f-92f6-fd8ebb4bf73c admin admin] RBAC: Authorizing `identity:delete_project(project_id=02ed9f0eac5849358cf11723e1a51fee)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:06.962825 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93025130-bab8-4d4f-92f6-fd8ebb4bf73c admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:07.101120 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:07.106399 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-93025130-bab8-4d4f-92f6-fd8ebb4bf73c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02ed9f0eac5849358cf11723e1a51fee', 'request_id': 'req-93025130-bab8-4d4f-92f6-fd8ebb4bf73c'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:07.216951 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-93025130-bab8-4d4f-92f6-fd8ebb4bf73c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02ed9f0eac5849358cf11723e1a51fee', 'request_id': 'req-93025130-bab8-4d4f-92f6-fd8ebb4bf73c'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:07.222045 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-93025130-bab8-4d4f-92f6-fd8ebb4bf73c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02ed9f0eac5849358cf11723e1a51fee', 'request_id': 'req-93025130-bab8-4d4f-92f6-fd8ebb4bf73c'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:07.240214 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-93025130-bab8-4d4f-92f6-fd8ebb4bf73c admin admin] The token cache is being invalidate because project 02ed9f0eac5849358cf11723e1a51fee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:09:07.240214 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-93025130-bab8-4d4f-92f6-fd8ebb4bf73c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:07.241154 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-93025130-bab8-4d4f-92f6-fd8ebb4bf73c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:07.246572 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1303/2609] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 09:09:06 2022] DELETE /identity/v3/projects/02ed9f0eac5849358cf11723e1a51fee => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:09:07.248110 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4aa142-a61e-4b06-a128-f61f62dd1910 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:07.248971 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4aa142-a61e-4b06-a128-f61f62dd1910 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:07.249252 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4aa142-a61e-4b06-a128-f61f62dd1910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:07.278660 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-4e4aa142-a61e-4b06-a128-f61f62dd1910 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:09:07.281477 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-4e4aa142-a61e-4b06-a128-f61f62dd1910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:07.356404 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1307/2610] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:09:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:09:07.364685 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e85ef2-55c7-4374-8127-2e115fda3fcf None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:07.365540 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e85ef2-55c7-4374-8127-2e115fda3fcf None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:07.365823 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e85ef2-55c7-4374-8127-2e115fda3fcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:07.388494 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-a6e85ef2-55c7-4374-8127-2e115fda3fcf None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:09:07.389833 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-a6e85ef2-55c7-4374-8127-2e115fda3fcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:07.401686 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1304/2611] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:09:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:09:07.402778 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4aa142-a61e-4b06-a128-f61f62dd1910 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:07.413979 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4aa142-a61e-4b06-a128-f61f62dd1910 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:07.419280 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e57d41-39ea-425e-b187-1cb22af5c4aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:07.420551 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e57d41-39ea-425e-b187-1cb22af5c4aa admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:07.420826 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e57d41-39ea-425e-b187-1cb22af5c4aa admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:07.421108 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e57d41-39ea-425e-b187-1cb22af5c4aa admin admin] PATH_INFO: `/v3/domains` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:07.421785 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e57d41-39ea-425e-b187-1cb22af5c4aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:09:07.422090 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e57d41-39ea-425e-b187-1cb22af5c4aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:07.423813 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e57d41-39ea-425e-b187-1cb22af5c4aa admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:07.436616 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1308/2612] 10.222.0.55 () {58 vars in 1153 bytes} [Wed Aug 24 09:09:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:09:07.441086 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e85ef2-55c7-4374-8127-2e115fda3fcf None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:07.450451 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e85ef2-55c7-4374-8127-2e115fda3fcf None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:07.462163 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a6b19d-7805-4de3-9bea-5ec43c336381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:07.462163 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a6b19d-7805-4de3-9bea-5ec43c336381 admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:07.462163 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a6b19d-7805-4de3-9bea-5ec43c336381 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:07.462163 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a6b19d-7805-4de3-9bea-5ec43c336381 admin admin] PATH_INFO: `/v3/projects` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:07.462664 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a6b19d-7805-4de3-9bea-5ec43c336381 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:07.465320 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a6b19d-7805-4de3-9bea-5ec43c336381 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:07.594323 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1305/2613] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 09:09:07 2022] POST /identity/v3/projects => generated 371 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:09:07.598842 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e57d41-39ea-425e-b187-1cb22af5c4aa admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:07.606057 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:07.619922 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e57d41-39ea-425e-b187-1cb22af5c4aa admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:07.692787 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a26dc85-d619-4dd4-9f2f-552624e9871b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:07.693434 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a26dc85-d619-4dd4-9f2f-552624e9871b admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:07.693711 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a26dc85-d619-4dd4-9f2f-552624e9871b admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:07.693992 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a26dc85-d619-4dd4-9f2f-552624e9871b admin admin] PATH_INFO: `/v3/users` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:07.694629 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a26dc85-d619-4dd4-9f2f-552624e9871b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:07.698239 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a26dc85-d619-4dd4-9f2f-552624e9871b admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:07.792330 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1309/2614] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 09:09:07 2022] POST /identity/v3/users => generated 286 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:09:07.799483 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a6b19d-7805-4de3-9bea-5ec43c336381 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:07.811247 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a6b19d-7805-4de3-9bea-5ec43c336381 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:07.816274 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ad90ea-6c5c-4444-80ee-ef47a91e0b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:07.816732 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ad90ea-6c5c-4444-80ee-ef47a91e0b8a admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:07.817143 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ad90ea-6c5c-4444-80ee-ef47a91e0b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:07.817553 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ad90ea-6c5c-4444-80ee-ef47a91e0b8a admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:07.818625 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ad90ea-6c5c-4444-80ee-ef47a91e0b8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:07.821299 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ad90ea-6c5c-4444-80ee-ef47a91e0b8a admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:07.833174 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1306/2615] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:09:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:09:07.837583 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a26dc85-d619-4dd4-9f2f-552624e9871b admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:07.849951 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a26dc85-d619-4dd4-9f2f-552624e9871b admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:07.855667 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693b2c6a-433d-4915-8ac8-bf845e5c13af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:07.856467 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693b2c6a-433d-4915-8ac8-bf845e5c13af admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:07.856874 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693b2c6a-433d-4915-8ac8-bf845e5c13af admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:07.857157 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693b2c6a-433d-4915-8ac8-bf845e5c13af admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:07.858045 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693b2c6a-433d-4915-8ac8-bf845e5c13af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:07.859785 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693b2c6a-433d-4915-8ac8-bf845e5c13af admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:07.872796 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1310/2616] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:09:07 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:09:07.877174 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ad90ea-6c5c-4444-80ee-ef47a91e0b8a admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:07.887311 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ad90ea-6c5c-4444-80ee-ef47a91e0b8a admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:07.892409 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688f4697-81ee-48c5-abfe-287b728a711a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:07.893110 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688f4697-81ee-48c5-abfe-287b728a711a admin admin] REQUEST_METHOD: `PUT` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:07.893386 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688f4697-81ee-48c5-abfe-287b728a711a admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:07.893656 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688f4697-81ee-48c5-abfe-287b728a711a admin admin] PATH_INFO: `/v3/projects/cf6a786aaf254d6fa32b32721b0b6645/users/8ecfe4a5ab414a9a9b80edb224c82699/roles/b674fd0a592e4e0cada811bba5ceea4d` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:07.914303 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688f4697-81ee-48c5-abfe-287b728a711a admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf6a786aaf254d6fa32b32721b0b6645, user_id=8ecfe4a5ab414a9a9b80edb224c82699, role_id=b674fd0a592e4e0cada811bba5ceea4d)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:07.917394 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688f4697-81ee-48c5-abfe-287b728a711a admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:07.994309 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1307/2617] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 09:09:07 2022] PUT /identity/v3/projects/cf6a786aaf254d6fa32b32721b0b6645/users/8ecfe4a5ab414a9a9b80edb224c82699/roles/b674fd0a592e4e0cada811bba5ceea4d => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:09:08.002452 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c2f1cc-3a14-4795-a17c-8d5ab578fa44 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:08.002739 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c2f1cc-3a14-4795-a17c-8d5ab578fa44 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:08.003016 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c2f1cc-3a14-4795-a17c-8d5ab578fa44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:08.048112 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-68c2f1cc-3a14-4795-a17c-8d5ab578fa44 None None] MFA Rules not processed for user `8ecfe4a5ab414a9a9b80edb224c82699`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:09:08.049834 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-68c2f1cc-3a14-4795-a17c-8d5ab578fa44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:08.140060 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1311/2618] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:09:08 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:09:08.140965 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688f4697-81ee-48c5-abfe-287b728a711a admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:08.221634 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688f4697-81ee-48c5-abfe-287b728a711a admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:08.227496 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97a2370-550f-482f-8f84-a69eb995c179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:08.233428 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97a2370-550f-482f-8f84-a69eb995c179 admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:08.233428 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97a2370-550f-482f-8f84-a69eb995c179 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:08.233428 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97a2370-550f-482f-8f84-a69eb995c179 admin admin] PATH_INFO: `/v3/projects` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:08.233428 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97a2370-550f-482f-8f84-a69eb995c179 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:08.233428 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97a2370-550f-482f-8f84-a69eb995c179 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:08.341003 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1308/2619] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 09:09:08 2022] POST /identity/v3/projects => generated 371 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:09:08.348793 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c2f1cc-3a14-4795-a17c-8d5ab578fa44 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:08.371724 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c2f1cc-3a14-4795-a17c-8d5ab578fa44 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:08.479134 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64701bef-4fdf-4bf5-ada5-3b7fe8334b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:08.479762 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64701bef-4fdf-4bf5-ada5-3b7fe8334b0d admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:08.480037 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64701bef-4fdf-4bf5-ada5-3b7fe8334b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:08.480309 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64701bef-4fdf-4bf5-ada5-3b7fe8334b0d admin admin] PATH_INFO: `/v3/users` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:08.481004 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64701bef-4fdf-4bf5-ada5-3b7fe8334b0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:08.483027 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64701bef-4fdf-4bf5-ada5-3b7fe8334b0d admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:08.603384 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:08.606278 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1312/2620] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 09:09:08 2022] POST /identity/v3/users => generated 292 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:09:08.607912 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97a2370-550f-482f-8f84-a69eb995c179 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:08.626929 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97a2370-550f-482f-8f84-a69eb995c179 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:08.631292 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ff394b-9f8f-4767-ba62-aaf4523748af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:08.631906 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ff394b-9f8f-4767-ba62-aaf4523748af admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:08.632187 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ff394b-9f8f-4767-ba62-aaf4523748af admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:08.632478 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ff394b-9f8f-4767-ba62-aaf4523748af admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:08.633158 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ff394b-9f8f-4767-ba62-aaf4523748af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:08.634823 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ff394b-9f8f-4767-ba62-aaf4523748af admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:08.646532 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1309/2621] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:09:08 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:09:08.650722 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64701bef-4fdf-4bf5-ada5-3b7fe8334b0d admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:08.661722 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64701bef-4fdf-4bf5-ada5-3b7fe8334b0d admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:08.667449 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f501d200-b64f-41a4-a5b7-c782f1bce7fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:08.668149 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f501d200-b64f-41a4-a5b7-c782f1bce7fb admin admin] REQUEST_METHOD: `PUT` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:08.668714 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f501d200-b64f-41a4-a5b7-c782f1bce7fb admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:08.669397 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f501d200-b64f-41a4-a5b7-c782f1bce7fb admin admin] PATH_INFO: `/v3/projects/c2655a5059b84131ba58bc83e2993036/users/c6bec127606547fcbf4a7b3756de4a9f/roles/b870cd1acaa744e1ae1d58b9ac283821` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:08.689462 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f501d200-b64f-41a4-a5b7-c782f1bce7fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2655a5059b84131ba58bc83e2993036, user_id=c6bec127606547fcbf4a7b3756de4a9f, role_id=b870cd1acaa744e1ae1d58b9ac283821)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:08.692376 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f501d200-b64f-41a4-a5b7-c782f1bce7fb admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:08.752180 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1313/2622] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 09:09:08 2022] PUT /identity/v3/projects/c2655a5059b84131ba58bc83e2993036/users/c6bec127606547fcbf4a7b3756de4a9f/roles/b870cd1acaa744e1ae1d58b9ac283821 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:09:08.757143 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ff394b-9f8f-4767-ba62-aaf4523748af admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:08.771179 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ff394b-9f8f-4767-ba62-aaf4523748af admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:08.821784 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:08.852402 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43679cbc-529d-4974-b93b-9340f3514a71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:08.853292 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43679cbc-529d-4974-b93b-9340f3514a71 admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:08.853575 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43679cbc-529d-4974-b93b-9340f3514a71 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:08.853853 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43679cbc-529d-4974-b93b-9340f3514a71 admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:08.854508 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43679cbc-529d-4974-b93b-9340f3514a71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:08.856111 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43679cbc-529d-4974-b93b-9340f3514a71 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:08.867822 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1310/2623] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:09:08 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:09:08.872117 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f501d200-b64f-41a4-a5b7-c782f1bce7fb admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:08.894186 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f501d200-b64f-41a4-a5b7-c782f1bce7fb admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:08.900630 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9fae26-4e57-40cf-b50a-72149aec90a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:08.901309 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9fae26-4e57-40cf-b50a-72149aec90a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:08.903899 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9fae26-4e57-40cf-b50a-72149aec90a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:08.903899 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9fae26-4e57-40cf-b50a-72149aec90a5 admin admin] PATH_INFO: `/v3/projects/c2655a5059b84131ba58bc83e2993036/users/c6bec127606547fcbf4a7b3756de4a9f/roles/b870cd1acaa744e1ae1d58b9ac283821` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:08.903899 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9fae26-4e57-40cf-b50a-72149aec90a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2655a5059b84131ba58bc83e2993036, user_id=c6bec127606547fcbf4a7b3756de4a9f, role_id=b870cd1acaa744e1ae1d58b9ac283821)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:08.906532 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9fae26-4e57-40cf-b50a-72149aec90a5 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:08.917695 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1314/2624] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 09:09:08 2022] PUT /identity/v3/projects/c2655a5059b84131ba58bc83e2993036/users/c6bec127606547fcbf4a7b3756de4a9f/roles/b870cd1acaa744e1ae1d58b9ac283821 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:09:08.932683 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b388a7de-d5c8-4388-add6-c21bb2dc1a10 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:08.932683 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b388a7de-d5c8-4388-add6-c21bb2dc1a10 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:08.932985 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b388a7de-d5c8-4388-add6-c21bb2dc1a10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:08.960982 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-b388a7de-d5c8-4388-add6-c21bb2dc1a10 None None] MFA Rules not processed for user `c6bec127606547fcbf4a7b3756de4a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:09:08.962205 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-b388a7de-d5c8-4388-add6-c21bb2dc1a10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:09.029504 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1311/2625] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:09:08 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:09:09.041382 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9fae26-4e57-40cf-b50a-72149aec90a5 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:09.043097 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-3a9fae26-4e57-40cf-b50a-72149aec90a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:09.053203 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b388a7de-d5c8-4388-add6-c21bb2dc1a10 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:09.055547 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-b388a7de-d5c8-4388-add6-c21bb2dc1a10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:09.103959 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9fae26-4e57-40cf-b50a-72149aec90a5 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:09.108218 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5563a9-0130-4845-a401-e3f00ec87d26 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbb9ac667f04c35a63b2689df18efed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:09.108858 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5563a9-0130-4845-a401-e3f00ec87d26 service cinder] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:09.109136 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5563a9-0130-4845-a401-e3f00ec87d26 service cinder] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:09.109408 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5563a9-0130-4845-a401-e3f00ec87d26 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:09.119176 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5563a9-0130-4845-a401-e3f00ec87d26 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:09.120968 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5563a9-0130-4845-a401-e3f00ec87d26 service cinder] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:09.123510 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b388a7de-d5c8-4388-add6-c21bb2dc1a10 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:09.128916 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65623884-813f-4e09-bf87-a3d1e75c51d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:09.129550 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65623884-813f-4e09-bf87-a3d1e75c51d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:09.129836 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65623884-813f-4e09-bf87-a3d1e75c51d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:09.130112 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65623884-813f-4e09-bf87-a3d1e75c51d6 admin admin] PATH_INFO: `/v3/users/1105970982014891902ebbb9904a57a5` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:09.131304 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65623884-813f-4e09-bf87-a3d1e75c51d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1105970982014891902ebbb9904a57a5)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:09.131598 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1315/2626] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:09:09 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:09:09.133739 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65623884-813f-4e09-bf87-a3d1e75c51d6 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:09.376253 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5563a9-0130-4845-a401-e3f00ec87d26 service cinder] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:09.385295 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5563a9-0130-4845-a401-e3f00ec87d26 service cinder] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:09.388960 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a73b85-9791-4b07-aa22-77d87f244f8a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbb9ac667f04c35a63b2689df18efed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:09.389557 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a73b85-9791-4b07-aa22-77d87f244f8a service cinder] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:09.389832 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a73b85-9791-4b07-aa22-77d87f244f8a service cinder] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:09.390096 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a73b85-9791-4b07-aa22-77d87f244f8a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:09.410818 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-65623884-813f-4e09-bf87-a3d1e75c51d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1105970982014891902ebbb9904a57a5'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:09.412105 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-65623884-813f-4e09-bf87-a3d1e75c51d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1105970982014891902ebbb9904a57a5'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:09.412844 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-65623884-813f-4e09-bf87-a3d1e75c51d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1105970982014891902ebbb9904a57a5'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:09.439294 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-65623884-813f-4e09-bf87-a3d1e75c51d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1105970982014891902ebbb9904a57a5'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:09.442053 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:09.453017 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a73b85-9791-4b07-aa22-77d87f244f8a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:09.454455 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a73b85-9791-4b07-aa22-77d87f244f8a service cinder] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:09.463461 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1316/2627] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:09:09 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 09:09:09.526621 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-65623884-813f-4e09-bf87-a3d1e75c51d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1105970982014891902ebbb9904a57a5'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:09.540583 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1312/2628] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 09:09:09 2022] DELETE /identity/v3/users/1105970982014891902ebbb9904a57a5 => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:09:09.780051 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a73b85-9791-4b07-aa22-77d87f244f8a service cinder] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:09.791920 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a73b85-9791-4b07-aa22-77d87f244f8a service cinder] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:09.860248 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1458eb2-4d1d-47ab-af79-85c8d690d787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:09.861093 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1458eb2-4d1d-47ab-af79-85c8d690d787 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:09.861516 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1458eb2-4d1d-47ab-af79-85c8d690d787 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:09.862180 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1458eb2-4d1d-47ab-af79-85c8d690d787 admin admin] PATH_INFO: `/v3/projects/a69447ca2f314084925442b7f8c9ee2e` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:09.863900 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1458eb2-4d1d-47ab-af79-85c8d690d787 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a69447ca2f314084925442b7f8c9ee2e)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:09.867881 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1458eb2-4d1d-47ab-af79-85c8d690d787 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:09.878022 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:10.018838 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-c1458eb2-4d1d-47ab-af79-85c8d690d787 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a69447ca2f314084925442b7f8c9ee2e', 'request_id': 'req-c1458eb2-4d1d-47ab-af79-85c8d690d787'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:10.027995 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-c1458eb2-4d1d-47ab-af79-85c8d690d787 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a69447ca2f314084925442b7f8c9ee2e', 'request_id': 'req-c1458eb2-4d1d-47ab-af79-85c8d690d787'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:10.115431 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-c1458eb2-4d1d-47ab-af79-85c8d690d787 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a69447ca2f314084925442b7f8c9ee2e', 'request_id': 'req-c1458eb2-4d1d-47ab-af79-85c8d690d787'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:10.129126 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-c1458eb2-4d1d-47ab-af79-85c8d690d787 admin admin] The token cache is being invalidate because project a69447ca2f314084925442b7f8c9ee2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:09:10.129415 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-c1458eb2-4d1d-47ab-af79-85c8d690d787 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:10.130091 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-c1458eb2-4d1d-47ab-af79-85c8d690d787 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:10.131571 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1317/2629] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 09:09:09 2022] DELETE /identity/v3/projects/a69447ca2f314084925442b7f8c9ee2e => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:09:10.136401 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cdaf94-4060-41ea-aa60-db6f742fb662 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:10.137015 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cdaf94-4060-41ea-aa60-db6f742fb662 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:10.137152 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cdaf94-4060-41ea-aa60-db6f742fb662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:10.156353 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-86cdaf94-4060-41ea-aa60-db6f742fb662 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:09:10.157615 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-86cdaf94-4060-41ea-aa60-db6f742fb662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:10.219002 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1313/2630] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:09:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:09:10.228001 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc7b1da-2cb4-4142-85ff-ea04b3020a74 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:10.228588 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc7b1da-2cb4-4142-85ff-ea04b3020a74 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:10.233455 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc7b1da-2cb4-4142-85ff-ea04b3020a74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:10.252202 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-2fc7b1da-2cb4-4142-85ff-ea04b3020a74 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:09:10.253491 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-2fc7b1da-2cb4-4142-85ff-ea04b3020a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:10.261732 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1318/2631] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:09:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:09:10.266170 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cdaf94-4060-41ea-aa60-db6f742fb662 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:10.277986 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cdaf94-4060-41ea-aa60-db6f742fb662 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:10.284373 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0de206-6784-40a0-b659-96b6abce23a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:10.285206 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0de206-6784-40a0-b659-96b6abce23a8 admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:10.285479 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0de206-6784-40a0-b659-96b6abce23a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:10.285745 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0de206-6784-40a0-b659-96b6abce23a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:10.286649 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0de206-6784-40a0-b659-96b6abce23a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:09:10.286947 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0de206-6784-40a0-b659-96b6abce23a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:10.288631 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0de206-6784-40a0-b659-96b6abce23a8 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:10.301525 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1314/2632] 10.222.0.55 () {58 vars in 1153 bytes} [Wed Aug 24 09:09:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:09:10.306014 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc7b1da-2cb4-4142-85ff-ea04b3020a74 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:10.315721 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc7b1da-2cb4-4142-85ff-ea04b3020a74 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:10.320474 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b4bac0-dd56-43bd-89d4-cbe3449ec971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:10.324851 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b4bac0-dd56-43bd-89d4-cbe3449ec971 admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:10.324851 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b4bac0-dd56-43bd-89d4-cbe3449ec971 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:10.324851 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b4bac0-dd56-43bd-89d4-cbe3449ec971 admin admin] PATH_INFO: `/v3/projects` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:10.324851 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b4bac0-dd56-43bd-89d4-cbe3449ec971 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:10.324851 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b4bac0-dd56-43bd-89d4-cbe3449ec971 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:10.419970 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1319/2633] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 09:09:10 2022] POST /identity/v3/projects => generated 403 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:09:10.424284 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0de206-6784-40a0-b659-96b6abce23a8 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:10.434378 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0de206-6784-40a0-b659-96b6abce23a8 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:10.495839 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c52efa-e48e-4413-817d-29e5e58c3a38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:10.496506 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c52efa-e48e-4413-817d-29e5e58c3a38 admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:10.496813 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c52efa-e48e-4413-817d-29e5e58c3a38 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:10.497080 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c52efa-e48e-4413-817d-29e5e58c3a38 admin admin] PATH_INFO: `/v3/users` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:10.498206 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c52efa-e48e-4413-817d-29e5e58c3a38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:10.500144 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c52efa-e48e-4413-817d-29e5e58c3a38 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:10.555877 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1315/2634] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 09:09:10 2022] POST /identity/v3/users => generated 302 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:09:10.560588 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b4bac0-dd56-43bd-89d4-cbe3449ec971 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:10.571309 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b4bac0-dd56-43bd-89d4-cbe3449ec971 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:10.576266 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02103623-8c74-44ce-a221-e43e46b92627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:10.576970 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02103623-8c74-44ce-a221-e43e46b92627 admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:10.577256 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02103623-8c74-44ce-a221-e43e46b92627 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:10.577532 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02103623-8c74-44ce-a221-e43e46b92627 admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:10.578180 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02103623-8c74-44ce-a221-e43e46b92627 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:10.579768 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02103623-8c74-44ce-a221-e43e46b92627 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:10.590310 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1320/2635] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:09:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:09:10.600211 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c52efa-e48e-4413-817d-29e5e58c3a38 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:10.605185 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c52efa-e48e-4413-817d-29e5e58c3a38 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:10.610830 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40b4ad0-fe7f-4643-8bbd-f3e75cb8313d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:10.611465 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40b4ad0-fe7f-4643-8bbd-f3e75cb8313d admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:10.611763 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40b4ad0-fe7f-4643-8bbd-f3e75cb8313d admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:10.612035 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40b4ad0-fe7f-4643-8bbd-f3e75cb8313d admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:10.612750 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40b4ad0-fe7f-4643-8bbd-f3e75cb8313d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:10.614405 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40b4ad0-fe7f-4643-8bbd-f3e75cb8313d admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:10.626983 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1316/2636] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:09:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:09:10.631378 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02103623-8c74-44ce-a221-e43e46b92627 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:10.641828 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02103623-8c74-44ce-a221-e43e46b92627 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:10.647474 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b3084d-59ac-48ba-97ab-c5765010da7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:10.648224 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b3084d-59ac-48ba-97ab-c5765010da7e admin admin] REQUEST_METHOD: `PUT` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:10.648490 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b3084d-59ac-48ba-97ab-c5765010da7e admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:10.648805 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b3084d-59ac-48ba-97ab-c5765010da7e admin admin] PATH_INFO: `/v3/projects/4a106cded6d445af9d44a36984171bac/users/283fb896141e4293bf7cabeb4deeed57/roles/b674fd0a592e4e0cada811bba5ceea4d` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:10.664626 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b3084d-59ac-48ba-97ab-c5765010da7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a106cded6d445af9d44a36984171bac, user_id=283fb896141e4293bf7cabeb4deeed57, role_id=b674fd0a592e4e0cada811bba5ceea4d)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:10.667356 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b3084d-59ac-48ba-97ab-c5765010da7e admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:10.739454 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1321/2637] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 09:09:10 2022] PUT /identity/v3/projects/4a106cded6d445af9d44a36984171bac/users/283fb896141e4293bf7cabeb4deeed57/roles/b674fd0a592e4e0cada811bba5ceea4d => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:09:10.750378 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1323be7-a03a-4d56-b968-68b06ab5f8d4 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:10.750658 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1323be7-a03a-4d56-b968-68b06ab5f8d4 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:10.750954 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1323be7-a03a-4d56-b968-68b06ab5f8d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:10.778546 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-a1323be7-a03a-4d56-b968-68b06ab5f8d4 None None] MFA Rules not processed for user `283fb896141e4293bf7cabeb4deeed57`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:09:10.779763 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-a1323be7-a03a-4d56-b968-68b06ab5f8d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:10.820987 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:10.844849 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1317/2638] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:09:10 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:09:10.849361 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b3084d-59ac-48ba-97ab-c5765010da7e admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:10.918498 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b3084d-59ac-48ba-97ab-c5765010da7e admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:10.925303 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d6ac9a-8431-4fda-8eb0-eecc9836db27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:10.925918 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d6ac9a-8431-4fda-8eb0-eecc9836db27 admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:10.926202 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d6ac9a-8431-4fda-8eb0-eecc9836db27 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:10.926477 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d6ac9a-8431-4fda-8eb0-eecc9836db27 admin admin] PATH_INFO: `/v3/projects` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:10.927146 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d6ac9a-8431-4fda-8eb0-eecc9836db27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:10.930032 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d6ac9a-8431-4fda-8eb0-eecc9836db27 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:11.082807 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1322/2639] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 09:09:10 2022] POST /identity/v3/projects => generated 403 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:09:11.083993 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1323be7-a03a-4d56-b968-68b06ab5f8d4 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:11.095437 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1323be7-a03a-4d56-b968-68b06ab5f8d4 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:11.160699 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1766db-53a3-43b6-aa0e-8f0ef57f04f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:11.161739 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1766db-53a3-43b6-aa0e-8f0ef57f04f3 admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:11.161955 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1766db-53a3-43b6-aa0e-8f0ef57f04f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:11.162739 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1766db-53a3-43b6-aa0e-8f0ef57f04f3 admin admin] PATH_INFO: `/v3/users` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:11.163596 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1766db-53a3-43b6-aa0e-8f0ef57f04f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:11.166595 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1766db-53a3-43b6-aa0e-8f0ef57f04f3 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:11.338500 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1318/2640] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 09:09:11 2022] POST /identity/v3/users => generated 308 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:09:11.343187 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d6ac9a-8431-4fda-8eb0-eecc9836db27 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:11.353618 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d6ac9a-8431-4fda-8eb0-eecc9836db27 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:11.358196 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b776e2-6192-4b24-a5a4-3072162ff276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:11.358804 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b776e2-6192-4b24-a5a4-3072162ff276 admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:11.359083 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b776e2-6192-4b24-a5a4-3072162ff276 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:11.359383 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b776e2-6192-4b24-a5a4-3072162ff276 admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:11.360017 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b776e2-6192-4b24-a5a4-3072162ff276 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:11.361647 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b776e2-6192-4b24-a5a4-3072162ff276 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:11.372395 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1323/2641] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:09:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:09:11.378766 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1766db-53a3-43b6-aa0e-8f0ef57f04f3 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:11.394407 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1766db-53a3-43b6-aa0e-8f0ef57f04f3 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:11.399253 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b42fe78-ef6c-47c9-911a-fb992dfde74c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:11.399906 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b42fe78-ef6c-47c9-911a-fb992dfde74c admin admin] REQUEST_METHOD: `PUT` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:11.400178 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b42fe78-ef6c-47c9-911a-fb992dfde74c admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:11.400449 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b42fe78-ef6c-47c9-911a-fb992dfde74c admin admin] PATH_INFO: `/v3/projects/3d2e7a620ff64c059c8c1d11f7a0912d/users/3c1df921ec644cac8667bea35b6b3a87/roles/b870cd1acaa744e1ae1d58b9ac283821` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:11.416011 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b42fe78-ef6c-47c9-911a-fb992dfde74c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d2e7a620ff64c059c8c1d11f7a0912d, user_id=3c1df921ec644cac8667bea35b6b3a87, role_id=b870cd1acaa744e1ae1d58b9ac283821)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:11.418729 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b42fe78-ef6c-47c9-911a-fb992dfde74c admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:11.510697 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1319/2642] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 09:09:11 2022] PUT /identity/v3/projects/3d2e7a620ff64c059c8c1d11f7a0912d/users/3c1df921ec644cac8667bea35b6b3a87/roles/b870cd1acaa744e1ae1d58b9ac283821 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:09:11.515740 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b776e2-6192-4b24-a5a4-3072162ff276 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:11.522279 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:11.526020 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b776e2-6192-4b24-a5a4-3072162ff276 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:11.581911 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbb86a1-ef61-433e-abf2-f8be8ff9a5f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:11.582515 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbb86a1-ef61-433e-abf2-f8be8ff9a5f4 admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:11.582792 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbb86a1-ef61-433e-abf2-f8be8ff9a5f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:11.583060 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbb86a1-ef61-433e-abf2-f8be8ff9a5f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:11.583777 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbb86a1-ef61-433e-abf2-f8be8ff9a5f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:11.585401 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbb86a1-ef61-433e-abf2-f8be8ff9a5f4 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:11.595799 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1324/2643] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:09:11 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:09:11.600107 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b42fe78-ef6c-47c9-911a-fb992dfde74c admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:11.610891 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b42fe78-ef6c-47c9-911a-fb992dfde74c admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:11.615929 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafe1634-d02e-4bce-b8eb-9458e09132b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:11.620888 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafe1634-d02e-4bce-b8eb-9458e09132b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:11.620888 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafe1634-d02e-4bce-b8eb-9458e09132b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:11.620888 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafe1634-d02e-4bce-b8eb-9458e09132b1 admin admin] PATH_INFO: `/v3/projects/3d2e7a620ff64c059c8c1d11f7a0912d/users/3c1df921ec644cac8667bea35b6b3a87/roles/b870cd1acaa744e1ae1d58b9ac283821` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:11.620888 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafe1634-d02e-4bce-b8eb-9458e09132b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d2e7a620ff64c059c8c1d11f7a0912d, user_id=3c1df921ec644cac8667bea35b6b3a87, role_id=b870cd1acaa744e1ae1d58b9ac283821)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:11.622054 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafe1634-d02e-4bce-b8eb-9458e09132b1 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:11.632900 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1320/2644] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 09:09:11 2022] PUT /identity/v3/projects/3d2e7a620ff64c059c8c1d11f7a0912d/users/3c1df921ec644cac8667bea35b6b3a87/roles/b870cd1acaa744e1ae1d58b9ac283821 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:09:11.645391 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db06718f-005c-48ec-aa42-629253573c6c None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:11.652171 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db06718f-005c-48ec-aa42-629253573c6c None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:11.652171 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db06718f-005c-48ec-aa42-629253573c6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:11.675200 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-db06718f-005c-48ec-aa42-629253573c6c None None] MFA Rules not processed for user `3c1df921ec644cac8667bea35b6b3a87`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:09:11.676407 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-db06718f-005c-48ec-aa42-629253573c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:11.751760 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1325/2645] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:09:11 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:09:11.762909 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafe1634-d02e-4bce-b8eb-9458e09132b1 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:11.765359 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-eafe1634-d02e-4bce-b8eb-9458e09132b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:11.828467 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafe1634-d02e-4bce-b8eb-9458e09132b1 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:11.832474 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e88733-c80c-44f4-b559-c3e692cf7dca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbb9ac667f04c35a63b2689df18efed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:11.833113 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e88733-c80c-44f4-b559-c3e692cf7dca service cinder] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:11.833386 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e88733-c80c-44f4-b559-c3e692cf7dca service cinder] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:11.833662 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e88733-c80c-44f4-b559-c3e692cf7dca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:11.843220 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e88733-c80c-44f4-b559-c3e692cf7dca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:11.844685 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e88733-c80c-44f4-b559-c3e692cf7dca service cinder] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:11.854450 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1321/2646] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:09:11 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:09:12.074901 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db06718f-005c-48ec-aa42-629253573c6c None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:12.148131 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db06718f-005c-48ec-aa42-629253573c6c None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:12.148131 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f200cd9e-d1f5-4401-b227-3d6291bc9aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:12.148131 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f200cd9e-d1f5-4401-b227-3d6291bc9aae admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:12.148131 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f200cd9e-d1f5-4401-b227-3d6291bc9aae admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:12.148720 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f200cd9e-d1f5-4401-b227-3d6291bc9aae admin admin] PATH_INFO: `/v3/users/283fb896141e4293bf7cabeb4deeed57` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:12.148720 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f200cd9e-d1f5-4401-b227-3d6291bc9aae admin admin] RBAC: Authorizing `identity:delete_user(user_id=283fb896141e4293bf7cabeb4deeed57)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:12.149168 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f200cd9e-d1f5-4401-b227-3d6291bc9aae admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:12.312218 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:12.426982 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-f200cd9e-d1f5-4401-b227-3d6291bc9aae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '283fb896141e4293bf7cabeb4deeed57'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:12.431154 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-f200cd9e-d1f5-4401-b227-3d6291bc9aae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '283fb896141e4293bf7cabeb4deeed57'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:12.431154 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-f200cd9e-d1f5-4401-b227-3d6291bc9aae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '283fb896141e4293bf7cabeb4deeed57'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:12.440708 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-f200cd9e-d1f5-4401-b227-3d6291bc9aae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '283fb896141e4293bf7cabeb4deeed57'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:12.509260 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-f200cd9e-d1f5-4401-b227-3d6291bc9aae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '283fb896141e4293bf7cabeb4deeed57'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:12.522848 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1326/2647] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 09:09:12 2022] DELETE /identity/v3/users/283fb896141e4293bf7cabeb4deeed57 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:09:12.527548 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b77c8d-fa5a-4a6e-9020-8bd0880c82e7 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:12.528010 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b77c8d-fa5a-4a6e-9020-8bd0880c82e7 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:12.528125 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b77c8d-fa5a-4a6e-9020-8bd0880c82e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:12.547833 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-a7b77c8d-fa5a-4a6e-9020-8bd0880c82e7 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:09:12.549059 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-a7b77c8d-fa5a-4a6e-9020-8bd0880c82e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:12.578774 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:12.607094 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1322/2648] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:09:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:09:12.616788 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f200cd9e-d1f5-4401-b227-3d6291bc9aae admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:12.618233 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-f200cd9e-d1f5-4401-b227-3d6291bc9aae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:12.677540 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f200cd9e-d1f5-4401-b227-3d6291bc9aae admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:12.681043 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9534d1a2-ca2e-4327-8bf7-b1ede422db72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:12.681643 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9534d1a2-ca2e-4327-8bf7-b1ede422db72 service neutron] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:12.681916 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9534d1a2-ca2e-4327-8bf7-b1ede422db72 service neutron] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:12.682183 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9534d1a2-ca2e-4327-8bf7-b1ede422db72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:12.691645 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9534d1a2-ca2e-4327-8bf7-b1ede422db72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:12.693089 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9534d1a2-ca2e-4327-8bf7-b1ede422db72 service neutron] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:12.703108 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1327/2649] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:09:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:09:13.347545 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b77c8d-fa5a-4a6e-9020-8bd0880c82e7 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:13.349132 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-a7b77c8d-fa5a-4a6e-9020-8bd0880c82e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:13.361259 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b77c8d-fa5a-4a6e-9020-8bd0880c82e7 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:13.367790 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8f64c4-f65c-494a-9021-6609a26e0504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:13.367790 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8f64c4-f65c-494a-9021-6609a26e0504 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:13.367790 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8f64c4-f65c-494a-9021-6609a26e0504 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:13.367790 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8f64c4-f65c-494a-9021-6609a26e0504 admin admin] PATH_INFO: `/v3/projects/4a106cded6d445af9d44a36984171bac` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:13.369034 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8f64c4-f65c-494a-9021-6609a26e0504 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a106cded6d445af9d44a36984171bac)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:13.371101 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8f64c4-f65c-494a-9021-6609a26e0504 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:13.532892 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-ba8f64c4-f65c-494a-9021-6609a26e0504 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a106cded6d445af9d44a36984171bac', 'request_id': 'req-ba8f64c4-f65c-494a-9021-6609a26e0504'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:13.635145 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-ba8f64c4-f65c-494a-9021-6609a26e0504 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a106cded6d445af9d44a36984171bac', 'request_id': 'req-ba8f64c4-f65c-494a-9021-6609a26e0504'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:13.643337 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-ba8f64c4-f65c-494a-9021-6609a26e0504 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a106cded6d445af9d44a36984171bac', 'request_id': 'req-ba8f64c4-f65c-494a-9021-6609a26e0504'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:13.656185 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-ba8f64c4-f65c-494a-9021-6609a26e0504 admin admin] The token cache is being invalidate because project 4a106cded6d445af9d44a36984171bac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:09:13.656452 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-ba8f64c4-f65c-494a-9021-6609a26e0504 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:13.657151 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-ba8f64c4-f65c-494a-9021-6609a26e0504 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:13.658629 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1323/2650] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 09:09:13 2022] DELETE /identity/v3/projects/4a106cded6d445af9d44a36984171bac => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:09:13.663286 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9534d1a2-ca2e-4327-8bf7-b1ede422db72 service neutron] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:13.675159 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9534d1a2-ca2e-4327-8bf7-b1ede422db72 service neutron] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:13.735424 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07001990-2711-4bd5-bc76-7f40c9e77a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:13.736144 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07001990-2711-4bd5-bc76-7f40c9e77a27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:13.740106 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07001990-2711-4bd5-bc76-7f40c9e77a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:13.740106 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07001990-2711-4bd5-bc76-7f40c9e77a27 admin admin] PATH_INFO: `/v3/users/3c1df921ec644cac8667bea35b6b3a87` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:13.740106 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07001990-2711-4bd5-bc76-7f40c9e77a27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c1df921ec644cac8667bea35b6b3a87)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:13.740106 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07001990-2711-4bd5-bc76-7f40c9e77a27 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:13.959576 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-07001990-2711-4bd5-bc76-7f40c9e77a27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c1df921ec644cac8667bea35b6b3a87'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:13.973647 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-07001990-2711-4bd5-bc76-7f40c9e77a27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c1df921ec644cac8667bea35b6b3a87'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:13.973647 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-07001990-2711-4bd5-bc76-7f40c9e77a27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c1df921ec644cac8667bea35b6b3a87'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:13.973647 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:13.982483 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-07001990-2711-4bd5-bc76-7f40c9e77a27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c1df921ec644cac8667bea35b6b3a87'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:14.109654 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-07001990-2711-4bd5-bc76-7f40c9e77a27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c1df921ec644cac8667bea35b6b3a87'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:14.123611 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1328/2651] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 09:09:13 2022] DELETE /identity/v3/users/3c1df921ec644cac8667bea35b6b3a87 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:09:14.127084 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8f64c4-f65c-494a-9021-6609a26e0504 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:14.128742 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-ba8f64c4-f65c-494a-9021-6609a26e0504 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:14.214702 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8f64c4-f65c-494a-9021-6609a26e0504 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:14.276841 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:14.283257 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cca91b6-4b87-4dc1-9d1a-320cc3c85a10 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf0c99771661413aaa0f237af6b394ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:14.283884 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cca91b6-4b87-4dc1-9d1a-320cc3c85a10 service placement] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:14.284144 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cca91b6-4b87-4dc1-9d1a-320cc3c85a10 service placement] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:14.284413 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cca91b6-4b87-4dc1-9d1a-320cc3c85a10 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:14.286453 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-8cca91b6-4b87-4dc1-9d1a-320cc3c85a10 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:14.298249 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cca91b6-4b87-4dc1-9d1a-320cc3c85a10 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:14.310276 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cca91b6-4b87-4dc1-9d1a-320cc3c85a10 service placement] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:14.310276 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1324/2652] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:09:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:09:14.592001 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07001990-2711-4bd5-bc76-7f40c9e77a27 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:14.593514 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-07001990-2711-4bd5-bc76-7f40c9e77a27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:14.689845 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07001990-2711-4bd5-bc76-7f40c9e77a27 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:14.698561 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda1a209-bdde-4fd1-8627-96cec44e9496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:14.699263 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda1a209-bdde-4fd1-8627-96cec44e9496 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:14.699541 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda1a209-bdde-4fd1-8627-96cec44e9496 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:14.699827 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda1a209-bdde-4fd1-8627-96cec44e9496 admin admin] PATH_INFO: `/v3/projects/3d2e7a620ff64c059c8c1d11f7a0912d` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:14.701182 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda1a209-bdde-4fd1-8627-96cec44e9496 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d2e7a620ff64c059c8c1d11f7a0912d)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:14.703111 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda1a209-bdde-4fd1-8627-96cec44e9496 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:14.839802 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-bda1a209-bdde-4fd1-8627-96cec44e9496 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d2e7a620ff64c059c8c1d11f7a0912d', 'request_id': 'req-bda1a209-bdde-4fd1-8627-96cec44e9496'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:14.849078 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-bda1a209-bdde-4fd1-8627-96cec44e9496 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d2e7a620ff64c059c8c1d11f7a0912d', 'request_id': 'req-bda1a209-bdde-4fd1-8627-96cec44e9496'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:14.927284 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-bda1a209-bdde-4fd1-8627-96cec44e9496 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d2e7a620ff64c059c8c1d11f7a0912d', 'request_id': 'req-bda1a209-bdde-4fd1-8627-96cec44e9496'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:14.942246 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-bda1a209-bdde-4fd1-8627-96cec44e9496 admin admin] The token cache is being invalidate because project 3d2e7a620ff64c059c8c1d11f7a0912d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:09:14.942525 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-bda1a209-bdde-4fd1-8627-96cec44e9496 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:14.943203 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-bda1a209-bdde-4fd1-8627-96cec44e9496 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:14.946110 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1329/2653] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 09:09:14 2022] DELETE /identity/v3/projects/3d2e7a620ff64c059c8c1d11f7a0912d => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:09:14.950339 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e37026b-5d35-4dfc-b9b2-97cd0c2fbf8a None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:14.950832 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e37026b-5d35-4dfc-b9b2-97cd0c2fbf8a None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:14.950983 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e37026b-5d35-4dfc-b9b2-97cd0c2fbf8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:14.974455 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-9e37026b-5d35-4dfc-b9b2-97cd0c2fbf8a None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:09:14.975734 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-9e37026b-5d35-4dfc-b9b2-97cd0c2fbf8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:15.041000 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1325/2654] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:09:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:09:15.049075 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9106e670-2d7e-4ccf-837d-d11b592d42fd None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:15.049447 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9106e670-2d7e-4ccf-837d-d11b592d42fd None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:15.049815 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9106e670-2d7e-4ccf-837d-d11b592d42fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:15.069218 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-9106e670-2d7e-4ccf-837d-d11b592d42fd None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:09:15.070412 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-9106e670-2d7e-4ccf-837d-d11b592d42fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:15.078798 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1330/2655] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:09:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:09:15.082987 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e37026b-5d35-4dfc-b9b2-97cd0c2fbf8a None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:15.093997 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e37026b-5d35-4dfc-b9b2-97cd0c2fbf8a None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:15.098485 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b0b62b-46ca-4060-9058-6995b1709565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:15.099091 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b0b62b-46ca-4060-9058-6995b1709565 admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:15.099364 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b0b62b-46ca-4060-9058-6995b1709565 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:15.099636 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b0b62b-46ca-4060-9058-6995b1709565 admin admin] PATH_INFO: `/v3/domains` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:15.100274 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b0b62b-46ca-4060-9058-6995b1709565 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:09:15.100604 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b0b62b-46ca-4060-9058-6995b1709565 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:15.102182 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b0b62b-46ca-4060-9058-6995b1709565 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:15.113900 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1326/2656] 10.222.0.55 () {58 vars in 1153 bytes} [Wed Aug 24 09:09:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:09:15.118532 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9106e670-2d7e-4ccf-837d-d11b592d42fd None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:15.128243 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9106e670-2d7e-4ccf-837d-d11b592d42fd None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:15.133793 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eb7b7b-8b3b-4d58-8571-781067db3fbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:15.135016 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eb7b7b-8b3b-4d58-8571-781067db3fbb admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:15.135016 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eb7b7b-8b3b-4d58-8571-781067db3fbb admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:15.135237 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eb7b7b-8b3b-4d58-8571-781067db3fbb admin admin] PATH_INFO: `/v3/projects` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:15.135724 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4eb7b7b-8b3b-4d58-8571-781067db3fbb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:15.137766 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4eb7b7b-8b3b-4d58-8571-781067db3fbb admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:15.237693 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1331/2657] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 09:09:15 2022] POST /identity/v3/projects => generated 381 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:09:15.242041 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b0b62b-46ca-4060-9058-6995b1709565 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:15.252085 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b0b62b-46ca-4060-9058-6995b1709565 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:15.309747 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424008d0-d4bc-405d-af4f-187d786520af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:15.310388 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424008d0-d4bc-405d-af4f-187d786520af admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:15.310661 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424008d0-d4bc-405d-af4f-187d786520af admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:15.310933 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424008d0-d4bc-405d-af4f-187d786520af admin admin] PATH_INFO: `/v3/users` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:15.311575 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424008d0-d4bc-405d-af4f-187d786520af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:15.313513 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424008d0-d4bc-405d-af4f-187d786520af admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:15.390460 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1327/2658] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 09:09:15 2022] POST /identity/v3/users => generated 291 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:09:15.394637 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eb7b7b-8b3b-4d58-8571-781067db3fbb admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:15.405310 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eb7b7b-8b3b-4d58-8571-781067db3fbb admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:15.410068 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf06f0bb-bec2-4ae7-8a95-0c2b1e96fdc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:15.410699 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf06f0bb-bec2-4ae7-8a95-0c2b1e96fdc1 admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:15.410974 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf06f0bb-bec2-4ae7-8a95-0c2b1e96fdc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:15.411526 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf06f0bb-bec2-4ae7-8a95-0c2b1e96fdc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:15.412218 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf06f0bb-bec2-4ae7-8a95-0c2b1e96fdc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:15.413854 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf06f0bb-bec2-4ae7-8a95-0c2b1e96fdc1 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:15.423979 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1332/2659] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:09:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:09:15.428569 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424008d0-d4bc-405d-af4f-187d786520af admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:15.439503 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424008d0-d4bc-405d-af4f-187d786520af admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:15.444203 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23845ca6-b19f-4979-88c5-9ab6dbc6c2e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:15.444828 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23845ca6-b19f-4979-88c5-9ab6dbc6c2e0 admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:15.445203 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23845ca6-b19f-4979-88c5-9ab6dbc6c2e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:15.445479 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23845ca6-b19f-4979-88c5-9ab6dbc6c2e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:15.446117 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23845ca6-b19f-4979-88c5-9ab6dbc6c2e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:15.447710 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23845ca6-b19f-4979-88c5-9ab6dbc6c2e0 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:15.459223 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1328/2660] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:09:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:09:15.463405 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf06f0bb-bec2-4ae7-8a95-0c2b1e96fdc1 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:15.473653 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf06f0bb-bec2-4ae7-8a95-0c2b1e96fdc1 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:15.478278 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c56cbc7-2e1a-4b35-a426-8aa5d2d85ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:15.478925 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c56cbc7-2e1a-4b35-a426-8aa5d2d85ea7 admin admin] REQUEST_METHOD: `PUT` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:15.479204 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c56cbc7-2e1a-4b35-a426-8aa5d2d85ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:15.479477 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c56cbc7-2e1a-4b35-a426-8aa5d2d85ea7 admin admin] PATH_INFO: `/v3/projects/ccf0ad74b2254dcabad98c9c089a389c/users/01ab48d3e59445d097e79e1381140fdf/roles/b674fd0a592e4e0cada811bba5ceea4d` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:15.494175 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c56cbc7-2e1a-4b35-a426-8aa5d2d85ea7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccf0ad74b2254dcabad98c9c089a389c, user_id=01ab48d3e59445d097e79e1381140fdf, role_id=b674fd0a592e4e0cada811bba5ceea4d)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:15.496977 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c56cbc7-2e1a-4b35-a426-8aa5d2d85ea7 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:15.499808 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:15.601362 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1333/2661] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 09:09:15 2022] PUT /identity/v3/projects/ccf0ad74b2254dcabad98c9c089a389c/users/01ab48d3e59445d097e79e1381140fdf/roles/b674fd0a592e4e0cada811bba5ceea4d => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:09:15.611768 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175b6de1-3d84-49ca-8b58-1408f48dd45e None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:15.612121 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175b6de1-3d84-49ca-8b58-1408f48dd45e None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:15.612362 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175b6de1-3d84-49ca-8b58-1408f48dd45e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:15.642838 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-175b6de1-3d84-49ca-8b58-1408f48dd45e None None] MFA Rules not processed for user `01ab48d3e59445d097e79e1381140fdf`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:09:15.644088 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-175b6de1-3d84-49ca-8b58-1408f48dd45e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:15.691100 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:15.717995 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1329/2662] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:09:15 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:09:15.722900 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c56cbc7-2e1a-4b35-a426-8aa5d2d85ea7 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:15.802305 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c56cbc7-2e1a-4b35-a426-8aa5d2d85ea7 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:15.807085 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b05308-ae5f-4963-828c-7544e3432f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:15.807696 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b05308-ae5f-4963-828c-7544e3432f8a admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:15.807973 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b05308-ae5f-4963-828c-7544e3432f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:15.808243 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b05308-ae5f-4963-828c-7544e3432f8a admin admin] PATH_INFO: `/v3/projects` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:15.808948 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b05308-ae5f-4963-828c-7544e3432f8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:15.810905 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b05308-ae5f-4963-828c-7544e3432f8a admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:15.920383 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1334/2663] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 09:09:15 2022] POST /identity/v3/projects => generated 379 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:09:15.922112 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175b6de1-3d84-49ca-8b58-1408f48dd45e None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:15.933196 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175b6de1-3d84-49ca-8b58-1408f48dd45e None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:15.994246 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cb5066-ed0a-433d-a11e-bc3775824c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:15.994898 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cb5066-ed0a-433d-a11e-bc3775824c75 admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:15.995205 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cb5066-ed0a-433d-a11e-bc3775824c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:15.995557 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cb5066-ed0a-433d-a11e-bc3775824c75 admin admin] PATH_INFO: `/v3/users` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:15.996264 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cb5066-ed0a-433d-a11e-bc3775824c75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:15.998354 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cb5066-ed0a-433d-a11e-bc3775824c75 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:16.018251 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af5cd12-d9ba-4a93-b3c4-b2c43598c47b None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:16.018581 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af5cd12-d9ba-4a93-b3c4-b2c43598c47b None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:16.018853 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af5cd12-d9ba-4a93-b3c4-b2c43598c47b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:16.038946 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-3af5cd12-d9ba-4a93-b3c4-b2c43598c47b None None] MFA Rules not processed for user `f652a368b1cc42ae8b6f6c7a6dff6f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:09:16.040161 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-3af5cd12-d9ba-4a93-b3c4-b2c43598c47b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:16.086389 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1330/2664] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 09:09:15 2022] POST /identity/v3/users => generated 296 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:09:16.090310 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cb5066-ed0a-433d-a11e-bc3775824c75 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:16.099832 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1335/2665] 10.222.0.55 () {58 vars in 1009 bytes} [Wed Aug 24 09:09:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:09:16.101008 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cb5066-ed0a-433d-a11e-bc3775824c75 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:16.105095 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1cbb59-4d1a-4de5-bcb9-a1ede58c0adb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:16.105696 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1cbb59-4d1a-4de5-bcb9-a1ede58c0adb admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:16.105982 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1cbb59-4d1a-4de5-bcb9-a1ede58c0adb admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:16.106256 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1cbb59-4d1a-4de5-bcb9-a1ede58c0adb admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:16.106897 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1cbb59-4d1a-4de5-bcb9-a1ede58c0adb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:16.109305 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1cbb59-4d1a-4de5-bcb9-a1ede58c0adb admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:16.122950 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1331/2666] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:09:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 09:09:16.124717 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af5cd12-d9ba-4a93-b3c4-b2c43598c47b None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:16.136476 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af5cd12-d9ba-4a93-b3c4-b2c43598c47b None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:16.141160 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f960505-a9ee-4812-8289-312be37094fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:16.141818 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f960505-a9ee-4812-8289-312be37094fc admin admin] REQUEST_METHOD: `PUT` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:16.142094 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f960505-a9ee-4812-8289-312be37094fc admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:16.142363 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f960505-a9ee-4812-8289-312be37094fc admin admin] PATH_INFO: `/v3/projects/b98e7f95768149a0b07d8e2668d8ffb3/users/cf6c9b9631444f1fa80ebb365109dba1/roles/b870cd1acaa744e1ae1d58b9ac283821` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:16.157659 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f960505-a9ee-4812-8289-312be37094fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=b98e7f95768149a0b07d8e2668d8ffb3, user_id=cf6c9b9631444f1fa80ebb365109dba1, role_id=b870cd1acaa744e1ae1d58b9ac283821)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:16.163014 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f960505-a9ee-4812-8289-312be37094fc admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:16.264217 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1336/2667] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 09:09:16 2022] PUT /identity/v3/projects/b98e7f95768149a0b07d8e2668d8ffb3/users/cf6c9b9631444f1fa80ebb365109dba1/roles/b870cd1acaa744e1ae1d58b9ac283821 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:09:16.268348 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1cbb59-4d1a-4de5-bcb9-a1ede58c0adb admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:16.278472 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1cbb59-4d1a-4de5-bcb9-a1ede58c0adb admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:16.291799 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f960505-a9ee-4812-8289-312be37094fc admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:16.293566 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-4f960505-a9ee-4812-8289-312be37094fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:16.336995 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c8968b-4817-484e-a8c8-cfee7fc6f535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:16.337601 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c8968b-4817-484e-a8c8-cfee7fc6f535 admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:16.337882 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c8968b-4817-484e-a8c8-cfee7fc6f535 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:16.338156 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c8968b-4817-484e-a8c8-cfee7fc6f535 admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:16.338805 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c8968b-4817-484e-a8c8-cfee7fc6f535 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:16.340384 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c8968b-4817-484e-a8c8-cfee7fc6f535 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:16.351208 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1332/2668] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:09:16 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:09:16.355430 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c8968b-4817-484e-a8c8-cfee7fc6f535 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:16.363035 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:16.365697 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c8968b-4817-484e-a8c8-cfee7fc6f535 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:16.369384 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f960505-a9ee-4812-8289-312be37094fc admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:16.370401 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1109d1-c89f-4665-8fac-ab5c99949d76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:16.371044 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1109d1-c89f-4665-8fac-ab5c99949d76 admin admin] REQUEST_METHOD: `PUT` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:16.377683 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1109d1-c89f-4665-8fac-ab5c99949d76 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:16.377683 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1109d1-c89f-4665-8fac-ab5c99949d76 admin admin] PATH_INFO: `/v3/projects/b98e7f95768149a0b07d8e2668d8ffb3/users/cf6c9b9631444f1fa80ebb365109dba1/roles/b870cd1acaa744e1ae1d58b9ac283821` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:16.377683 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1109d1-c89f-4665-8fac-ab5c99949d76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b98e7f95768149a0b07d8e2668d8ffb3, user_id=cf6c9b9631444f1fa80ebb365109dba1, role_id=b870cd1acaa744e1ae1d58b9ac283821)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:16.377968 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9a0ce1-8014-44a3-aba2-68982b88c3ae service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0ebb5ccabef45378f0c1da69721e051', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:16.377968 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9a0ce1-8014-44a3-aba2-68982b88c3ae service swift] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:16.377968 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9a0ce1-8014-44a3-aba2-68982b88c3ae service swift] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:16.377968 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9a0ce1-8014-44a3-aba2-68982b88c3ae service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:16.378349 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1109d1-c89f-4665-8fac-ab5c99949d76 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:16.387297 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1333/2669] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 09:09:16 2022] PUT /identity/v3/projects/b98e7f95768149a0b07d8e2668d8ffb3/users/cf6c9b9631444f1fa80ebb365109dba1/roles/b870cd1acaa744e1ae1d58b9ac283821 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:09:16.398413 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788b0419-4b8e-436f-9336-93ed8a8ecb3d None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:16.398694 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788b0419-4b8e-436f-9336-93ed8a8ecb3d None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:16.398966 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788b0419-4b8e-436f-9336-93ed8a8ecb3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:16.427817 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-788b0419-4b8e-436f-9336-93ed8a8ecb3d None None] MFA Rules not processed for user `cf6c9b9631444f1fa80ebb365109dba1`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:09:16.429180 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-788b0419-4b8e-436f-9336-93ed8a8ecb3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:16.436088 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9a0ce1-8014-44a3-aba2-68982b88c3ae service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:16.437822 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9a0ce1-8014-44a3-aba2-68982b88c3ae service swift] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:16.446952 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1337/2670] 10.222.0.55 () {60 vars in 1384 bytes} [Wed Aug 24 09:09:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 09:09:16.469765 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:16.501018 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1334/2671] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:09:16 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:09:16.512818 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9a0ce1-8014-44a3-aba2-68982b88c3ae service swift] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:16.515228 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-7e9a0ce1-8014-44a3-aba2-68982b88c3ae service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:16.575262 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9a0ce1-8014-44a3-aba2-68982b88c3ae service swift] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:16.626859 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac2a069-15f7-4ded-b8e4-e4e18b573e76 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbb9ac667f04c35a63b2689df18efed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:16.627416 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac2a069-15f7-4ded-b8e4-e4e18b573e76 service cinder] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:16.627692 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac2a069-15f7-4ded-b8e4-e4e18b573e76 service cinder] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:16.627967 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac2a069-15f7-4ded-b8e4-e4e18b573e76 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:16.636999 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac2a069-15f7-4ded-b8e4-e4e18b573e76 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:16.638411 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac2a069-15f7-4ded-b8e4-e4e18b573e76 service cinder] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:16.648289 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1338/2672] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:09:16 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:09:16.855150 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788b0419-4b8e-436f-9336-93ed8a8ecb3d None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:16.856752 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-788b0419-4b8e-436f-9336-93ed8a8ecb3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:16.871521 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788b0419-4b8e-436f-9336-93ed8a8ecb3d None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:16.880710 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf40fc2-eab4-47b5-a40a-a449003028d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbb9ac667f04c35a63b2689df18efed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:16.881321 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf40fc2-eab4-47b5-a40a-a449003028d9 service cinder] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:16.881596 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf40fc2-eab4-47b5-a40a-a449003028d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:16.882066 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf40fc2-eab4-47b5-a40a-a449003028d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:16.948932 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf40fc2-eab4-47b5-a40a-a449003028d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:16.950443 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf40fc2-eab4-47b5-a40a-a449003028d9 service cinder] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:16.960993 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1335/2673] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:09:16 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:09:17.278620 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed968bf-f268-4c0f-bf9e-f53aa1081006 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:17.279146 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed968bf-f268-4c0f-bf9e-f53aa1081006 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:17.279146 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed968bf-f268-4c0f-bf9e-f53aa1081006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:17.299547 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-bed968bf-f268-4c0f-bf9e-f53aa1081006 None None] MFA Rules not processed for user `f652a368b1cc42ae8b6f6c7a6dff6f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:09:17.300838 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-bed968bf-f268-4c0f-bf9e-f53aa1081006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:17.367627 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1339/2674] 10.222.0.55 () {58 vars in 967 bytes} [Wed Aug 24 09:09:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:09:17.384041 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf40fc2-eab4-47b5-a40a-a449003028d9 service cinder] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:17.448130 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf40fc2-eab4-47b5-a40a-a449003028d9 service cinder] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:17.453293 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d7f80a-81b2-4326-b775-02998b8d9472 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0ebb5ccabef45378f0c1da69721e051', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:17.453887 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d7f80a-81b2-4326-b775-02998b8d9472 service swift] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:17.454161 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d7f80a-81b2-4326-b775-02998b8d9472 service swift] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:17.454428 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d7f80a-81b2-4326-b775-02998b8d9472 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:17.462576 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d7f80a-81b2-4326-b775-02998b8d9472 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:17.464127 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d7f80a-81b2-4326-b775-02998b8d9472 service swift] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:17.471992 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1336/2675] 10.222.0.55 () {60 vars in 1384 bytes} [Wed Aug 24 09:09:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 09:09:17.988429 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b022b968-e7b4-449c-8631-dba8f7ffe10a None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:17.988741 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b022b968-e7b4-449c-8631-dba8f7ffe10a None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:17.989034 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b022b968-e7b4-449c-8631-dba8f7ffe10a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:17.997271 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:18.008115 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-b022b968-e7b4-449c-8631-dba8f7ffe10a None None] MFA Rules not processed for user `f652a368b1cc42ae8b6f6c7a6dff6f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:09:18.009569 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-b022b968-e7b4-449c-8631-dba8f7ffe10a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:18.018493 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1340/2676] 10.222.0.55 () {58 vars in 1009 bytes} [Wed Aug 24 09:09:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:09:18.030040 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d7f80a-81b2-4326-b775-02998b8d9472 service swift] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:18.041415 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d7f80a-81b2-4326-b775-02998b8d9472 service swift] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:18.046448 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c59c6f-c2eb-4132-a7b4-cb6c8a7e7501 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0ebb5ccabef45378f0c1da69721e051', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:18.047032 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c59c6f-c2eb-4132-a7b4-cb6c8a7e7501 service swift] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:18.047307 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c59c6f-c2eb-4132-a7b4-cb6c8a7e7501 service swift] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:18.047574 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c59c6f-c2eb-4132-a7b4-cb6c8a7e7501 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:18.055718 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c59c6f-c2eb-4132-a7b4-cb6c8a7e7501 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:18.057308 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c59c6f-c2eb-4132-a7b4-cb6c8a7e7501 service swift] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:18.065601 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1337/2677] 10.222.0.55 () {60 vars in 1384 bytes} [Wed Aug 24 09:09:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 09:09:18.576825 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93240e82-fa7a-4b1d-bc32-8ec123dd61fc None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:18.577866 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93240e82-fa7a-4b1d-bc32-8ec123dd61fc None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:18.577986 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93240e82-fa7a-4b1d-bc32-8ec123dd61fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:18.596977 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-93240e82-fa7a-4b1d-bc32-8ec123dd61fc None None] MFA Rules not processed for user `f652a368b1cc42ae8b6f6c7a6dff6f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:09:18.598103 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-93240e82-fa7a-4b1d-bc32-8ec123dd61fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:18.605234 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1341/2678] 10.222.0.55 () {58 vars in 967 bytes} [Wed Aug 24 09:09:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:09:18.617208 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c59c6f-c2eb-4132-a7b4-cb6c8a7e7501 service swift] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:18.628464 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c59c6f-c2eb-4132-a7b4-cb6c8a7e7501 service swift] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:18.633524 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33332ee8-3de6-4f2a-accf-2bdbfcaf5b54 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0ebb5ccabef45378f0c1da69721e051', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:18.634108 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33332ee8-3de6-4f2a-accf-2bdbfcaf5b54 service swift] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:18.634386 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33332ee8-3de6-4f2a-accf-2bdbfcaf5b54 service swift] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:18.634661 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33332ee8-3de6-4f2a-accf-2bdbfcaf5b54 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:18.643298 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33332ee8-3de6-4f2a-accf-2bdbfcaf5b54 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:18.644914 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33332ee8-3de6-4f2a-accf-2bdbfcaf5b54 service swift] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:18.653091 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1338/2679] 10.222.0.55 () {60 vars in 1384 bytes} [Wed Aug 24 09:09:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 09:09:21.731034 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93240e82-fa7a-4b1d-bc32-8ec123dd61fc None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:21.732435 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-93240e82-fa7a-4b1d-bc32-8ec123dd61fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:21.800693 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93240e82-fa7a-4b1d-bc32-8ec123dd61fc None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:21.805423 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f70572a-715e-458e-9b95-f0f9d416de3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:21.806066 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f70572a-715e-458e-9b95-f0f9d416de3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:21.806346 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f70572a-715e-458e-9b95-f0f9d416de3c admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:21.806625 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f70572a-715e-458e-9b95-f0f9d416de3c admin admin] PATH_INFO: `/v3/users/5c030e83e1cc40609d901de3ed82a90e` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:21.807735 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f70572a-715e-458e-9b95-f0f9d416de3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c030e83e1cc40609d901de3ed82a90e)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:21.809676 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f70572a-715e-458e-9b95-f0f9d416de3c admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:22.119348 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-8f70572a-715e-458e-9b95-f0f9d416de3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c030e83e1cc40609d901de3ed82a90e'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:22.120067 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-8f70572a-715e-458e-9b95-f0f9d416de3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c030e83e1cc40609d901de3ed82a90e'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:22.120704 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-8f70572a-715e-458e-9b95-f0f9d416de3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c030e83e1cc40609d901de3ed82a90e'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:22.135796 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-8f70572a-715e-458e-9b95-f0f9d416de3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c030e83e1cc40609d901de3ed82a90e'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:22.263209 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-8f70572a-715e-458e-9b95-f0f9d416de3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c030e83e1cc40609d901de3ed82a90e'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:22.277560 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1342/2680] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 09:09:21 2022] DELETE /identity/v3/users/5c030e83e1cc40609d901de3ed82a90e => generated 0 bytes in 547 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:09:22.282088 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b208184-504e-40da-a44f-8dc86218f798 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:22.282573 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b208184-504e-40da-a44f-8dc86218f798 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:22.282712 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b208184-504e-40da-a44f-8dc86218f798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:22.302358 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-1b208184-504e-40da-a44f-8dc86218f798 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:09:22.303607 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-1b208184-504e-40da-a44f-8dc86218f798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:22.336850 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:22.365250 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1339/2681] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:09:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:09:22.375166 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f70572a-715e-458e-9b95-f0f9d416de3c admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:22.376811 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-8f70572a-715e-458e-9b95-f0f9d416de3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:22.437403 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f70572a-715e-458e-9b95-f0f9d416de3c admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:22.441103 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3a8df9-fceb-466d-a1c1-7c35c25cfb54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:22.441713 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3a8df9-fceb-466d-a1c1-7c35c25cfb54 service neutron] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:22.441991 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3a8df9-fceb-466d-a1c1-7c35c25cfb54 service neutron] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:22.442262 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3a8df9-fceb-466d-a1c1-7c35c25cfb54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:22.451708 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3a8df9-fceb-466d-a1c1-7c35c25cfb54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:22.453183 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3a8df9-fceb-466d-a1c1-7c35c25cfb54 service neutron] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:22.463365 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1343/2682] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:09:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:09:22.716735 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b208184-504e-40da-a44f-8dc86218f798 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:22.719855 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-1b208184-504e-40da-a44f-8dc86218f798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:22.733640 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b208184-504e-40da-a44f-8dc86218f798 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:22.739070 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3349280-e834-4497-9c47-8ff2c62033a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:22.739881 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3349280-e834-4497-9c47-8ff2c62033a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:22.740175 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3349280-e834-4497-9c47-8ff2c62033a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:22.740490 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3349280-e834-4497-9c47-8ff2c62033a4 admin admin] PATH_INFO: `/v3/projects/bccb7b3c5c2844eb84a115fa60b186ff` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:22.742748 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3349280-e834-4497-9c47-8ff2c62033a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bccb7b3c5c2844eb84a115fa60b186ff)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:22.743727 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3349280-e834-4497-9c47-8ff2c62033a4 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:22.859888 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-c3349280-e834-4497-9c47-8ff2c62033a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bccb7b3c5c2844eb84a115fa60b186ff', 'request_id': 'req-c3349280-e834-4497-9c47-8ff2c62033a4'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:23.121165 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-c3349280-e834-4497-9c47-8ff2c62033a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bccb7b3c5c2844eb84a115fa60b186ff', 'request_id': 'req-c3349280-e834-4497-9c47-8ff2c62033a4'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:23.129624 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-c3349280-e834-4497-9c47-8ff2c62033a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bccb7b3c5c2844eb84a115fa60b186ff', 'request_id': 'req-c3349280-e834-4497-9c47-8ff2c62033a4'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:23.140361 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-c3349280-e834-4497-9c47-8ff2c62033a4 admin admin] The token cache is being invalidate because project bccb7b3c5c2844eb84a115fa60b186ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:09:23.140678 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-c3349280-e834-4497-9c47-8ff2c62033a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:23.148272 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-c3349280-e834-4497-9c47-8ff2c62033a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:23.148272 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1340/2683] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 09:09:22 2022] DELETE /identity/v3/projects/bccb7b3c5c2844eb84a115fa60b186ff => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:09:32.459010 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=83,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:32.850239 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3a8df9-fceb-466d-a1c1-7c35c25cfb54 service neutron] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:32.851654 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-4e3a8df9-fceb-466d-a1c1-7c35c25cfb54 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:32.865068 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3a8df9-fceb-466d-a1c1-7c35c25cfb54 service neutron] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:32.923279 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae06e14a-e1e8-4a76-a738-0dc69ea2a78c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:32.923911 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae06e14a-e1e8-4a76-a738-0dc69ea2a78c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:32.924188 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae06e14a-e1e8-4a76-a738-0dc69ea2a78c admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:32.924456 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae06e14a-e1e8-4a76-a738-0dc69ea2a78c admin admin] PATH_INFO: `/v3/users/01ab48d3e59445d097e79e1381140fdf` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:32.926146 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae06e14a-e1e8-4a76-a738-0dc69ea2a78c admin admin] RBAC: Authorizing `identity:delete_user(user_id=01ab48d3e59445d097e79e1381140fdf)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:32.928129 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae06e14a-e1e8-4a76-a738-0dc69ea2a78c admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:33.072349 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-ae06e14a-e1e8-4a76-a738-0dc69ea2a78c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01ab48d3e59445d097e79e1381140fdf'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:33.073021 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-ae06e14a-e1e8-4a76-a738-0dc69ea2a78c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01ab48d3e59445d097e79e1381140fdf'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:33.073727 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-ae06e14a-e1e8-4a76-a738-0dc69ea2a78c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01ab48d3e59445d097e79e1381140fdf'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:33.086053 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-ae06e14a-e1e8-4a76-a738-0dc69ea2a78c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01ab48d3e59445d097e79e1381140fdf'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:33.126067 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-ae06e14a-e1e8-4a76-a738-0dc69ea2a78c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01ab48d3e59445d097e79e1381140fdf'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:33.137239 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=38,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:33.139366 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1344/2684] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 09:09:32 2022] DELETE /identity/v3/users/01ab48d3e59445d097e79e1381140fdf => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:09:33.144031 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593ee336-2446-4091-86ea-733ba1e69838 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:33.144322 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593ee336-2446-4091-86ea-733ba1e69838 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:33.144655 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593ee336-2446-4091-86ea-733ba1e69838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:33.163622 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-593ee336-2446-4091-86ea-733ba1e69838 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:09:33.169802 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-593ee336-2446-4091-86ea-733ba1e69838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:33.225220 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1341/2685] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:09:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:09:33.235016 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae06e14a-e1e8-4a76-a738-0dc69ea2a78c admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:33.236433 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-ae06e14a-e1e8-4a76-a738-0dc69ea2a78c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:33.294583 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae06e14a-e1e8-4a76-a738-0dc69ea2a78c admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:33.298315 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5848ea-5b3c-4a2e-aead-33f53c60275f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:33.298954 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5848ea-5b3c-4a2e-aead-33f53c60275f service neutron] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:33.299233 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5848ea-5b3c-4a2e-aead-33f53c60275f service neutron] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:33.299507 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5848ea-5b3c-4a2e-aead-33f53c60275f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:33.309219 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5848ea-5b3c-4a2e-aead-33f53c60275f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:33.310644 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5848ea-5b3c-4a2e-aead-33f53c60275f service neutron] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:33.320428 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1345/2686] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:09:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:09:33.716274 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593ee336-2446-4091-86ea-733ba1e69838 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:33.717799 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-593ee336-2446-4091-86ea-733ba1e69838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:33.730622 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593ee336-2446-4091-86ea-733ba1e69838 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:33.735842 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94035828-84da-4137-a15a-ea04581174c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:33.736477 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94035828-84da-4137-a15a-ea04581174c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:33.736802 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94035828-84da-4137-a15a-ea04581174c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:33.737077 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94035828-84da-4137-a15a-ea04581174c1 admin admin] PATH_INFO: `/v3/projects/ccf0ad74b2254dcabad98c9c089a389c` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:33.738200 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94035828-84da-4137-a15a-ea04581174c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ccf0ad74b2254dcabad98c9c089a389c)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:33.740175 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94035828-84da-4137-a15a-ea04581174c1 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:33.849886 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-94035828-84da-4137-a15a-ea04581174c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ccf0ad74b2254dcabad98c9c089a389c', 'request_id': 'req-94035828-84da-4137-a15a-ea04581174c1'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:33.978308 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-94035828-84da-4137-a15a-ea04581174c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ccf0ad74b2254dcabad98c9c089a389c', 'request_id': 'req-94035828-84da-4137-a15a-ea04581174c1'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:33.985979 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-94035828-84da-4137-a15a-ea04581174c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ccf0ad74b2254dcabad98c9c089a389c', 'request_id': 'req-94035828-84da-4137-a15a-ea04581174c1'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:33.997322 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-94035828-84da-4137-a15a-ea04581174c1 admin admin] The token cache is being invalidate because project ccf0ad74b2254dcabad98c9c089a389c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:09:33.997639 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-94035828-84da-4137-a15a-ea04581174c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:33.998295 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-94035828-84da-4137-a15a-ea04581174c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:33.999775 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1342/2687] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 09:09:33 2022] DELETE /identity/v3/projects/ccf0ad74b2254dcabad98c9c089a389c => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:09:34.003992 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5848ea-5b3c-4a2e-aead-33f53c60275f service neutron] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:34.013895 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5848ea-5b3c-4a2e-aead-33f53c60275f service neutron] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:34.033417 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:34.072511 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532eaf98-16ea-4d70-ac1a-644bcc02998a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:34.073161 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532eaf98-16ea-4d70-ac1a-644bcc02998a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:34.073441 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532eaf98-16ea-4d70-ac1a-644bcc02998a admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:34.073713 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532eaf98-16ea-4d70-ac1a-644bcc02998a admin admin] PATH_INFO: `/v3/users/cf6c9b9631444f1fa80ebb365109dba1` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:34.074893 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532eaf98-16ea-4d70-ac1a-644bcc02998a admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf6c9b9631444f1fa80ebb365109dba1)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:34.076915 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532eaf98-16ea-4d70-ac1a-644bcc02998a admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:34.243164 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-532eaf98-16ea-4d70-ac1a-644bcc02998a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf6c9b9631444f1fa80ebb365109dba1'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:34.243800 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-532eaf98-16ea-4d70-ac1a-644bcc02998a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf6c9b9631444f1fa80ebb365109dba1'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:34.244396 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-532eaf98-16ea-4d70-ac1a-644bcc02998a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf6c9b9631444f1fa80ebb365109dba1'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:34.257030 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-532eaf98-16ea-4d70-ac1a-644bcc02998a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf6c9b9631444f1fa80ebb365109dba1'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:34.333149 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-532eaf98-16ea-4d70-ac1a-644bcc02998a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf6c9b9631444f1fa80ebb365109dba1'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:34.346233 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1346/2688] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 09:09:34 2022] DELETE /identity/v3/users/cf6c9b9631444f1fa80ebb365109dba1 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:09:34.871624 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94035828-84da-4137-a15a-ea04581174c1 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:34.873299 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-94035828-84da-4137-a15a-ea04581174c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:34.889758 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94035828-84da-4137-a15a-ea04581174c1 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:34.948609 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719f97c1-f665-4cf5-a0ef-e196ad7f713b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:34.949241 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719f97c1-f665-4cf5-a0ef-e196ad7f713b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:34.949522 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719f97c1-f665-4cf5-a0ef-e196ad7f713b admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:34.949797 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719f97c1-f665-4cf5-a0ef-e196ad7f713b admin admin] PATH_INFO: `/v3/projects/b98e7f95768149a0b07d8e2668d8ffb3` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:34.950952 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719f97c1-f665-4cf5-a0ef-e196ad7f713b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b98e7f95768149a0b07d8e2668d8ffb3)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:34.953056 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719f97c1-f665-4cf5-a0ef-e196ad7f713b admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:35.037697 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:35.053957 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-719f97c1-f665-4cf5-a0ef-e196ad7f713b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b98e7f95768149a0b07d8e2668d8ffb3', 'request_id': 'req-719f97c1-f665-4cf5-a0ef-e196ad7f713b'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:35.149052 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-719f97c1-f665-4cf5-a0ef-e196ad7f713b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b98e7f95768149a0b07d8e2668d8ffb3', 'request_id': 'req-719f97c1-f665-4cf5-a0ef-e196ad7f713b'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:35.155862 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-719f97c1-f665-4cf5-a0ef-e196ad7f713b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b98e7f95768149a0b07d8e2668d8ffb3', 'request_id': 'req-719f97c1-f665-4cf5-a0ef-e196ad7f713b'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:35.167585 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-719f97c1-f665-4cf5-a0ef-e196ad7f713b admin admin] The token cache is being invalidate because project b98e7f95768149a0b07d8e2668d8ffb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:09:35.167870 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-719f97c1-f665-4cf5-a0ef-e196ad7f713b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:35.168511 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-719f97c1-f665-4cf5-a0ef-e196ad7f713b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:35.170023 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1343/2689] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 09:09:34 2022] DELETE /identity/v3/projects/b98e7f95768149a0b07d8e2668d8ffb3 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:09:35.175030 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910d0cd2-fe9f-4d68-8518-29fc26ff4ebf None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:35.175510 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910d0cd2-fe9f-4d68-8518-29fc26ff4ebf None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:35.175611 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910d0cd2-fe9f-4d68-8518-29fc26ff4ebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:35.195364 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-910d0cd2-fe9f-4d68-8518-29fc26ff4ebf None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:09:35.196608 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-910d0cd2-fe9f-4d68-8518-29fc26ff4ebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:35.257018 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1347/2690] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:09:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:09:35.265059 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611f7d07-c606-4ae4-a1f8-bda18fe79b6a None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:35.265341 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611f7d07-c606-4ae4-a1f8-bda18fe79b6a None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:35.265613 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611f7d07-c606-4ae4-a1f8-bda18fe79b6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:35.286614 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-611f7d07-c606-4ae4-a1f8-bda18fe79b6a None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:09:35.287924 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-611f7d07-c606-4ae4-a1f8-bda18fe79b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:35.295893 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1344/2691] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:09:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:09:35.300365 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910d0cd2-fe9f-4d68-8518-29fc26ff4ebf None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:35.310119 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910d0cd2-fe9f-4d68-8518-29fc26ff4ebf None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:35.316320 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d6c975-092b-4334-a544-616a8094dfa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:35.316962 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d6c975-092b-4334-a544-616a8094dfa6 admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:35.317245 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d6c975-092b-4334-a544-616a8094dfa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:35.317561 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d6c975-092b-4334-a544-616a8094dfa6 admin admin] PATH_INFO: `/v3/domains` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:35.318208 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d6c975-092b-4334-a544-616a8094dfa6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:09:35.318508 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d6c975-092b-4334-a544-616a8094dfa6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:35.320105 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d6c975-092b-4334-a544-616a8094dfa6 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:35.332557 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1348/2692] 10.222.0.55 () {58 vars in 1153 bytes} [Wed Aug 24 09:09:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:09:35.336948 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611f7d07-c606-4ae4-a1f8-bda18fe79b6a None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:35.347281 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611f7d07-c606-4ae4-a1f8-bda18fe79b6a None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:35.352146 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb18e87-1005-4698-afeb-0dda4217bf43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:35.352826 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb18e87-1005-4698-afeb-0dda4217bf43 admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:35.353104 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb18e87-1005-4698-afeb-0dda4217bf43 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:35.353377 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb18e87-1005-4698-afeb-0dda4217bf43 admin admin] PATH_INFO: `/v3/projects` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:35.354041 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb18e87-1005-4698-afeb-0dda4217bf43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:35.355936 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb18e87-1005-4698-afeb-0dda4217bf43 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:35.472552 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1345/2693] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 09:09:35 2022] POST /identity/v3/projects => generated 363 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:09:35.476860 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d6c975-092b-4334-a544-616a8094dfa6 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:35.487209 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d6c975-092b-4334-a544-616a8094dfa6 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:35.514090 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:35.549303 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a3fde4-1d7c-46a5-a948-c4d24ee2926a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:35.555375 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a3fde4-1d7c-46a5-a948-c4d24ee2926a admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:35.555375 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a3fde4-1d7c-46a5-a948-c4d24ee2926a admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:35.555375 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a3fde4-1d7c-46a5-a948-c4d24ee2926a admin admin] PATH_INFO: `/v3/users` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:35.555375 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a3fde4-1d7c-46a5-a948-c4d24ee2926a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:35.555375 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a3fde4-1d7c-46a5-a948-c4d24ee2926a admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:35.652329 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1349/2694] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 09:09:35 2022] POST /identity/v3/users => generated 282 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:09:35.656518 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb18e87-1005-4698-afeb-0dda4217bf43 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:35.667154 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb18e87-1005-4698-afeb-0dda4217bf43 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:35.672592 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539ea149-fbb8-4077-98c0-4c9c33d23060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:35.673186 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539ea149-fbb8-4077-98c0-4c9c33d23060 admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:35.673462 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539ea149-fbb8-4077-98c0-4c9c33d23060 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:35.673730 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539ea149-fbb8-4077-98c0-4c9c33d23060 admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:35.674364 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539ea149-fbb8-4077-98c0-4c9c33d23060 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:35.675933 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539ea149-fbb8-4077-98c0-4c9c33d23060 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:35.686086 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1346/2695] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:09:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:09:35.690097 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a3fde4-1d7c-46a5-a948-c4d24ee2926a admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:35.700792 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a3fde4-1d7c-46a5-a948-c4d24ee2926a admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:35.705495 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25283bda-29dc-4c0a-bde4-ea12d2cbc557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:35.706747 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25283bda-29dc-4c0a-bde4-ea12d2cbc557 admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:35.707025 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25283bda-29dc-4c0a-bde4-ea12d2cbc557 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:35.707306 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25283bda-29dc-4c0a-bde4-ea12d2cbc557 admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:35.707937 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25283bda-29dc-4c0a-bde4-ea12d2cbc557 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:35.709554 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25283bda-29dc-4c0a-bde4-ea12d2cbc557 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:35.719546 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1350/2696] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:09:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:09:35.723987 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539ea149-fbb8-4077-98c0-4c9c33d23060 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:35.734465 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539ea149-fbb8-4077-98c0-4c9c33d23060 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:35.739254 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eecbeb-ff00-4ada-91a5-e9e0f566728b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:35.740510 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eecbeb-ff00-4ada-91a5-e9e0f566728b admin admin] REQUEST_METHOD: `PUT` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:35.740510 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eecbeb-ff00-4ada-91a5-e9e0f566728b admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:35.740510 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eecbeb-ff00-4ada-91a5-e9e0f566728b admin admin] PATH_INFO: `/v3/projects/8b51424604f6485489629751ef226b9a/users/7f5e83a49a3a4047bdcc2a72d4b374c2/roles/b674fd0a592e4e0cada811bba5ceea4d` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:35.755670 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2eecbeb-ff00-4ada-91a5-e9e0f566728b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b51424604f6485489629751ef226b9a, user_id=7f5e83a49a3a4047bdcc2a72d4b374c2, role_id=b674fd0a592e4e0cada811bba5ceea4d)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:35.758475 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2eecbeb-ff00-4ada-91a5-e9e0f566728b admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:35.846985 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1347/2697] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 09:09:35 2022] PUT /identity/v3/projects/8b51424604f6485489629751ef226b9a/users/7f5e83a49a3a4047bdcc2a72d4b374c2/roles/b674fd0a592e4e0cada811bba5ceea4d => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:09:35.857517 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67052faa-37ec-452a-abbd-62abe6fc8986 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:35.857801 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67052faa-37ec-452a-abbd-62abe6fc8986 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:35.858082 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67052faa-37ec-452a-abbd-62abe6fc8986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:35.889500 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-67052faa-37ec-452a-abbd-62abe6fc8986 None None] MFA Rules not processed for user `7f5e83a49a3a4047bdcc2a72d4b374c2`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:09:35.890996 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-67052faa-37ec-452a-abbd-62abe6fc8986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:35.963159 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1351/2698] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:09:35 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:09:35.967767 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eecbeb-ff00-4ada-91a5-e9e0f566728b admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:36.031731 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eecbeb-ff00-4ada-91a5-e9e0f566728b admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:36.036762 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd71edc-3025-4ce9-ae1b-c44109affb14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:36.037358 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd71edc-3025-4ce9-ae1b-c44109affb14 admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:36.037633 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd71edc-3025-4ce9-ae1b-c44109affb14 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:36.037902 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd71edc-3025-4ce9-ae1b-c44109affb14 admin admin] PATH_INFO: `/v3/projects` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:36.038525 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd71edc-3025-4ce9-ae1b-c44109affb14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:36.040419 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd71edc-3025-4ce9-ae1b-c44109affb14 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:36.121956 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1348/2699] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 09:09:35 2022] POST /identity/v3/projects => generated 365 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:09:36.126006 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67052faa-37ec-452a-abbd-62abe6fc8986 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:36.137897 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67052faa-37ec-452a-abbd-62abe6fc8986 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:36.194357 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ad1f0e-7223-40d2-9c57-4975ef9569d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:36.195001 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ad1f0e-7223-40d2-9c57-4975ef9569d8 admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:36.195226 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ad1f0e-7223-40d2-9c57-4975ef9569d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:36.195503 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ad1f0e-7223-40d2-9c57-4975ef9569d8 admin admin] PATH_INFO: `/v3/users` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:36.196129 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ad1f0e-7223-40d2-9c57-4975ef9569d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:36.198083 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ad1f0e-7223-40d2-9c57-4975ef9569d8 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:36.264257 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1352/2700] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 09:09:36 2022] POST /identity/v3/users => generated 289 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:09:36.268411 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd71edc-3025-4ce9-ae1b-c44109affb14 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:36.279101 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd71edc-3025-4ce9-ae1b-c44109affb14 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:36.284712 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366cf29b-84cc-4ce9-854a-d5bb522ba33a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:36.284951 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366cf29b-84cc-4ce9-854a-d5bb522ba33a admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:36.285228 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366cf29b-84cc-4ce9-854a-d5bb522ba33a admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:36.286300 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366cf29b-84cc-4ce9-854a-d5bb522ba33a admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:36.286300 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366cf29b-84cc-4ce9-854a-d5bb522ba33a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:36.288020 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366cf29b-84cc-4ce9-854a-d5bb522ba33a admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:36.298678 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1349/2701] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:09:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:09:36.303415 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ad1f0e-7223-40d2-9c57-4975ef9569d8 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:36.313513 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ad1f0e-7223-40d2-9c57-4975ef9569d8 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:36.318215 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d91103a-3f96-4e6c-aff3-fcc307e2b73e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:36.318859 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d91103a-3f96-4e6c-aff3-fcc307e2b73e admin admin] REQUEST_METHOD: `PUT` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:36.319137 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d91103a-3f96-4e6c-aff3-fcc307e2b73e admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:36.319410 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d91103a-3f96-4e6c-aff3-fcc307e2b73e admin admin] PATH_INFO: `/v3/projects/09cd7bbbe8a34a5b8865e07014fbfed4/users/d0252c38b33948649756baa09de1fb37/roles/b870cd1acaa744e1ae1d58b9ac283821` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:36.333991 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d91103a-3f96-4e6c-aff3-fcc307e2b73e admin admin] RBAC: Authorizing `identity:create_grant(project_id=09cd7bbbe8a34a5b8865e07014fbfed4, user_id=d0252c38b33948649756baa09de1fb37, role_id=b870cd1acaa744e1ae1d58b9ac283821)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:36.336657 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d91103a-3f96-4e6c-aff3-fcc307e2b73e admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:36.400690 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1353/2702] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 09:09:36 2022] PUT /identity/v3/projects/09cd7bbbe8a34a5b8865e07014fbfed4/users/d0252c38b33948649756baa09de1fb37/roles/b870cd1acaa744e1ae1d58b9ac283821 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:09:36.404447 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366cf29b-84cc-4ce9-854a-d5bb522ba33a admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:36.415375 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366cf29b-84cc-4ce9-854a-d5bb522ba33a admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:36.435431 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:36.471974 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab9bbf6-315a-40e5-9ea1-b22955cbcf98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:36.472614 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab9bbf6-315a-40e5-9ea1-b22955cbcf98 admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:36.472915 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab9bbf6-315a-40e5-9ea1-b22955cbcf98 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:36.473168 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab9bbf6-315a-40e5-9ea1-b22955cbcf98 admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:36.473836 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab9bbf6-315a-40e5-9ea1-b22955cbcf98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:36.475481 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab9bbf6-315a-40e5-9ea1-b22955cbcf98 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:36.485301 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1350/2703] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:09:36 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:09:36.490181 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d91103a-3f96-4e6c-aff3-fcc307e2b73e admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:36.495691 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:36.501226 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d91103a-3f96-4e6c-aff3-fcc307e2b73e admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:36.506461 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312208da-c1c6-40bb-b69b-3e2402beb959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:36.507120 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312208da-c1c6-40bb-b69b-3e2402beb959 admin admin] REQUEST_METHOD: `PUT` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:36.507394 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312208da-c1c6-40bb-b69b-3e2402beb959 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:36.507665 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312208da-c1c6-40bb-b69b-3e2402beb959 admin admin] PATH_INFO: `/v3/projects/09cd7bbbe8a34a5b8865e07014fbfed4/users/d0252c38b33948649756baa09de1fb37/roles/b870cd1acaa744e1ae1d58b9ac283821` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:36.509643 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312208da-c1c6-40bb-b69b-3e2402beb959 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09cd7bbbe8a34a5b8865e07014fbfed4, user_id=d0252c38b33948649756baa09de1fb37, role_id=b870cd1acaa744e1ae1d58b9ac283821)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:36.512374 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312208da-c1c6-40bb-b69b-3e2402beb959 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:36.523127 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1354/2704] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 09:09:36 2022] PUT /identity/v3/projects/09cd7bbbe8a34a5b8865e07014fbfed4/users/d0252c38b33948649756baa09de1fb37/roles/b870cd1acaa744e1ae1d58b9ac283821 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:09:36.534838 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacf10b9-64de-4ae1-83e0-a07fa9173aa0 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:36.536682 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacf10b9-64de-4ae1-83e0-a07fa9173aa0 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:36.536682 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacf10b9-64de-4ae1-83e0-a07fa9173aa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:36.564069 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-aacf10b9-64de-4ae1-83e0-a07fa9173aa0 None None] MFA Rules not processed for user `d0252c38b33948649756baa09de1fb37`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:09:36.565333 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-aacf10b9-64de-4ae1-83e0-a07fa9173aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:36.635804 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1351/2705] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:09:36 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:09:36.648160 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312208da-c1c6-40bb-b69b-3e2402beb959 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:36.650133 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-312208da-c1c6-40bb-b69b-3e2402beb959 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:36.711909 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312208da-c1c6-40bb-b69b-3e2402beb959 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:36.715553 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67da94e-c04e-404a-a62f-0221ab88b6cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbb9ac667f04c35a63b2689df18efed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:36.716189 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67da94e-c04e-404a-a62f-0221ab88b6cf service cinder] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:36.716433 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67da94e-c04e-404a-a62f-0221ab88b6cf service cinder] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:36.716738 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67da94e-c04e-404a-a62f-0221ab88b6cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:36.728058 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67da94e-c04e-404a-a62f-0221ab88b6cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:36.729670 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67da94e-c04e-404a-a62f-0221ab88b6cf service cinder] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:36.740080 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1355/2706] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:09:36 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 09:09:36.952129 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacf10b9-64de-4ae1-83e0-a07fa9173aa0 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:36.962194 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacf10b9-64de-4ae1-83e0-a07fa9173aa0 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:36.965734 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc012409-9539-47a8-be3b-217b6fadc3db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbb9ac667f04c35a63b2689df18efed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:36.966324 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc012409-9539-47a8-be3b-217b6fadc3db service cinder] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:36.966601 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc012409-9539-47a8-be3b-217b6fadc3db service cinder] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:36.966871 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc012409-9539-47a8-be3b-217b6fadc3db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:37.028786 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc012409-9539-47a8-be3b-217b6fadc3db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:37.030376 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc012409-9539-47a8-be3b-217b6fadc3db service cinder] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:37.040320 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1352/2707] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:09:36 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:09:46.736058 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:47.035740 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=80 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:54.592188 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67da94e-c04e-404a-a62f-0221ab88b6cf service cinder] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:54.593648 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-e67da94e-c04e-404a-a62f-0221ab88b6cf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:54.655839 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67da94e-c04e-404a-a62f-0221ab88b6cf service cinder] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:54.659523 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361b4b47-f6fb-4ca0-b3a6-b3b9222dd1b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:54.660135 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361b4b47-f6fb-4ca0-b3a6-b3b9222dd1b8 service neutron] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:54.660418 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361b4b47-f6fb-4ca0-b3a6-b3b9222dd1b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:54.660724 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361b4b47-f6fb-4ca0-b3a6-b3b9222dd1b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:54.662725 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-361b4b47-f6fb-4ca0-b3a6-b3b9222dd1b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:54.749051 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361b4b47-f6fb-4ca0-b3a6-b3b9222dd1b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:54.750369 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361b4b47-f6fb-4ca0-b3a6-b3b9222dd1b8 service neutron] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:54.769498 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1356/2708] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:09:54 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 09:10:03.523805 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1550b203-05ba-4818-aa3a-4f01907f372f None None] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:03.524389 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1550b203-05ba-4818-aa3a-4f01907f372f None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:03.524635 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1550b203-05ba-4818-aa3a-4f01907f372f None None] PATH_INFO: `/` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:03.526699 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1353/2709] 10.222.0.55 () {54 vars in 921 bytes} [Wed Aug 24 09:10:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:10:03.533322 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081039c5-ab36-4d7c-95da-a8153fdad4ad None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:03.533843 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081039c5-ab36-4d7c-95da-a8153fdad4ad None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:03.534020 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081039c5-ab36-4d7c-95da-a8153fdad4ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:03.601841 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-081039c5-ab36-4d7c-95da-a8153fdad4ad None None] MFA Rules not processed for user `28b58388d1664510b0b507323bca5ae8`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:10:03.603096 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-081039c5-ab36-4d7c-95da-a8153fdad4ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:03.609949 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1357/2710] 10.222.0.55 () {58 vars in 1018 bytes} [Wed Aug 24 09:10:03 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:10:03.622086 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1550b203-05ba-4818-aa3a-4f01907f372f None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:03.623850 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-1550b203-05ba-4818-aa3a-4f01907f372f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:03.638098 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1550b203-05ba-4818-aa3a-4f01907f372f None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:03.643182 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bb8185-d7a0-4589-9f39-850097254207 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:03.643182 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bb8185-d7a0-4589-9f39-850097254207 service neutron] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:03.643182 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bb8185-d7a0-4589-9f39-850097254207 service neutron] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:03.643182 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bb8185-d7a0-4589-9f39-850097254207 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:03.651429 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bb8185-d7a0-4589-9f39-850097254207 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:03.652876 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bb8185-d7a0-4589-9f39-850097254207 service neutron] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:03.661695 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1354/2711] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:10:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:10:03.750425 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ae33b4-9ba3-4710-bac7-acd050c0f541 None None] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:03.750747 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ae33b4-9ba3-4710-bac7-acd050c0f541 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:03.751156 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ae33b4-9ba3-4710-bac7-acd050c0f541 None None] PATH_INFO: `/` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:03.752861 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1358/2712] 10.222.0.55 () {54 vars in 921 bytes} [Wed Aug 24 09:10:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:10:03.758802 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1df0ff-ce65-4b4a-8372-aa15478664e0 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:03.759092 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1df0ff-ce65-4b4a-8372-aa15478664e0 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:03.759398 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1df0ff-ce65-4b4a-8372-aa15478664e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:03.806443 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-7b1df0ff-ce65-4b4a-8372-aa15478664e0 None None] MFA Rules not processed for user `f17da357e555420799ee681bffbcc268`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:10:03.807357 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-7b1df0ff-ce65-4b4a-8372-aa15478664e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:03.891834 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1355/2713] 10.222.0.55 () {58 vars in 1018 bytes} [Wed Aug 24 09:10:03 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:10:03.903191 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ae33b4-9ba3-4710-bac7-acd050c0f541 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:03.914476 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ae33b4-9ba3-4710-bac7-acd050c0f541 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:03.918080 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018d9267-c336-4f40-9bb2-6405e962019f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:03.918710 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018d9267-c336-4f40-9bb2-6405e962019f service neutron] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:03.918997 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018d9267-c336-4f40-9bb2-6405e962019f service neutron] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:03.919263 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018d9267-c336-4f40-9bb2-6405e962019f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:03.929856 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018d9267-c336-4f40-9bb2-6405e962019f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:03.931349 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018d9267-c336-4f40-9bb2-6405e962019f service neutron] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:03.942671 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1359/2714] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:10:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:10:05.196811 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1df0ff-ce65-4b4a-8372-aa15478664e0 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:05.261275 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1df0ff-ce65-4b4a-8372-aa15478664e0 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:05.266295 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dc2242-60a3-4c1a-a18d-0f92054cc308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:05.267026 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dc2242-60a3-4c1a-a18d-0f92054cc308 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:05.267326 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dc2242-60a3-4c1a-a18d-0f92054cc308 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:05.267595 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dc2242-60a3-4c1a-a18d-0f92054cc308 admin admin] PATH_INFO: `/v3/users/7f5e83a49a3a4047bdcc2a72d4b374c2` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:05.268877 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dc2242-60a3-4c1a-a18d-0f92054cc308 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f5e83a49a3a4047bdcc2a72d4b374c2)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:05.270799 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dc2242-60a3-4c1a-a18d-0f92054cc308 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:05.438324 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-a1dc2242-60a3-4c1a-a18d-0f92054cc308 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f5e83a49a3a4047bdcc2a72d4b374c2'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:05.439022 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-a1dc2242-60a3-4c1a-a18d-0f92054cc308 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f5e83a49a3a4047bdcc2a72d4b374c2'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:05.439667 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-a1dc2242-60a3-4c1a-a18d-0f92054cc308 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f5e83a49a3a4047bdcc2a72d4b374c2'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:05.453863 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-a1dc2242-60a3-4c1a-a18d-0f92054cc308 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f5e83a49a3a4047bdcc2a72d4b374c2'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:05.457613 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:10:05.566230 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-a1dc2242-60a3-4c1a-a18d-0f92054cc308 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f5e83a49a3a4047bdcc2a72d4b374c2'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:05.580710 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1356/2715] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 09:10:05 2022] DELETE /identity/v3/users/7f5e83a49a3a4047bdcc2a72d4b374c2 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:10:05.585380 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdcfee1-d4ee-4621-b6d3-338da2853571 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:05.585899 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdcfee1-d4ee-4621-b6d3-338da2853571 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:05.586034 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdcfee1-d4ee-4621-b6d3-338da2853571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:05.612934 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-dfdcfee1-d4ee-4621-b6d3-338da2853571 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:10:05.614107 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-dfdcfee1-d4ee-4621-b6d3-338da2853571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:05.682941 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=100 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:10:05.695111 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1360/2716] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:10:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:10:05.705773 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dc2242-60a3-4c1a-a18d-0f92054cc308 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:05.707515 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-a1dc2242-60a3-4c1a-a18d-0f92054cc308 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:05.767011 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dc2242-60a3-4c1a-a18d-0f92054cc308 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:05.770792 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210f610c-124c-44ef-b185-73a7bc6ad417 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:05.771565 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210f610c-124c-44ef-b185-73a7bc6ad417 service neutron] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:05.771706 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210f610c-124c-44ef-b185-73a7bc6ad417 service neutron] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:05.771967 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210f610c-124c-44ef-b185-73a7bc6ad417 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:05.781760 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210f610c-124c-44ef-b185-73a7bc6ad417 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:05.783205 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210f610c-124c-44ef-b185-73a7bc6ad417 service neutron] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:05.794122 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1357/2717] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:10:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:10:06.220425 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdcfee1-d4ee-4621-b6d3-338da2853571 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:06.221885 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-dfdcfee1-d4ee-4621-b6d3-338da2853571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:06.234792 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdcfee1-d4ee-4621-b6d3-338da2853571 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:06.239502 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a543323-7650-4e89-8b8e-823a909680c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:06.240183 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a543323-7650-4e89-8b8e-823a909680c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:06.240463 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a543323-7650-4e89-8b8e-823a909680c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:06.240791 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a543323-7650-4e89-8b8e-823a909680c6 admin admin] PATH_INFO: `/v3/projects/8b51424604f6485489629751ef226b9a` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:06.242016 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a543323-7650-4e89-8b8e-823a909680c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b51424604f6485489629751ef226b9a)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:06.243954 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a543323-7650-4e89-8b8e-823a909680c6 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:06.374823 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-9a543323-7650-4e89-8b8e-823a909680c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b51424604f6485489629751ef226b9a', 'request_id': 'req-9a543323-7650-4e89-8b8e-823a909680c6'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:06.382036 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-9a543323-7650-4e89-8b8e-823a909680c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b51424604f6485489629751ef226b9a', 'request_id': 'req-9a543323-7650-4e89-8b8e-823a909680c6'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:06.428969 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-9a543323-7650-4e89-8b8e-823a909680c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b51424604f6485489629751ef226b9a', 'request_id': 'req-9a543323-7650-4e89-8b8e-823a909680c6'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:06.442073 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-9a543323-7650-4e89-8b8e-823a909680c6 admin admin] The token cache is being invalidate because project 8b51424604f6485489629751ef226b9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:10:06.442286 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-9a543323-7650-4e89-8b8e-823a909680c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:06.442937 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-9a543323-7650-4e89-8b8e-823a909680c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:06.444591 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1361/2718] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 09:10:06 2022] DELETE /identity/v3/projects/8b51424604f6485489629751ef226b9a => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:10:06.448827 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210f610c-124c-44ef-b185-73a7bc6ad417 service neutron] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:06.459124 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210f610c-124c-44ef-b185-73a7bc6ad417 service neutron] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:06.520027 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aa1f8e-812b-47b2-bbf3-bc9373ec11a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:06.520609 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aa1f8e-812b-47b2-bbf3-bc9373ec11a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:06.520906 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aa1f8e-812b-47b2-bbf3-bc9373ec11a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:06.521170 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aa1f8e-812b-47b2-bbf3-bc9373ec11a5 admin admin] PATH_INFO: `/v3/users/d0252c38b33948649756baa09de1fb37` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:06.522329 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64aa1f8e-812b-47b2-bbf3-bc9373ec11a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0252c38b33948649756baa09de1fb37)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:06.524256 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64aa1f8e-812b-47b2-bbf3-bc9373ec11a5 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:06.783684 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-64aa1f8e-812b-47b2-bbf3-bc9373ec11a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0252c38b33948649756baa09de1fb37'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:06.784845 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-64aa1f8e-812b-47b2-bbf3-bc9373ec11a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0252c38b33948649756baa09de1fb37'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:06.785463 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-64aa1f8e-812b-47b2-bbf3-bc9373ec11a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0252c38b33948649756baa09de1fb37'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:06.803626 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-64aa1f8e-812b-47b2-bbf3-bc9373ec11a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0252c38b33948649756baa09de1fb37'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:06.891752 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-64aa1f8e-812b-47b2-bbf3-bc9373ec11a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0252c38b33948649756baa09de1fb37'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:06.909290 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1358/2719] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 09:10:06 2022] DELETE /identity/v3/users/d0252c38b33948649756baa09de1fb37 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:10:07.518618 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a543323-7650-4e89-8b8e-823a909680c6 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:07.521546 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-9a543323-7650-4e89-8b8e-823a909680c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:07.559570 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a543323-7650-4e89-8b8e-823a909680c6 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:07.673340 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40391608-32b1-4442-a610-9937b380b4da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:07.674007 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40391608-32b1-4442-a610-9937b380b4da admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:07.674282 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40391608-32b1-4442-a610-9937b380b4da admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:07.675160 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40391608-32b1-4442-a610-9937b380b4da admin admin] PATH_INFO: `/v3/projects/09cd7bbbe8a34a5b8865e07014fbfed4` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:07.676529 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40391608-32b1-4442-a610-9937b380b4da admin admin] RBAC: Authorizing `identity:delete_project(project_id=09cd7bbbe8a34a5b8865e07014fbfed4)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:07.682326 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40391608-32b1-4442-a610-9937b380b4da admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:07.822347 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-40391608-32b1-4442-a610-9937b380b4da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09cd7bbbe8a34a5b8865e07014fbfed4', 'request_id': 'req-40391608-32b1-4442-a610-9937b380b4da'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:07.829509 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-40391608-32b1-4442-a610-9937b380b4da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09cd7bbbe8a34a5b8865e07014fbfed4', 'request_id': 'req-40391608-32b1-4442-a610-9937b380b4da'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:07.914815 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-40391608-32b1-4442-a610-9937b380b4da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09cd7bbbe8a34a5b8865e07014fbfed4', 'request_id': 'req-40391608-32b1-4442-a610-9937b380b4da'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:07.944014 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-40391608-32b1-4442-a610-9937b380b4da admin admin] The token cache is being invalidate because project 09cd7bbbe8a34a5b8865e07014fbfed4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:10:07.944301 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-40391608-32b1-4442-a610-9937b380b4da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:07.944902 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-40391608-32b1-4442-a610-9937b380b4da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:07.947015 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1362/2720] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 09:10:07 2022] DELETE /identity/v3/projects/09cd7bbbe8a34a5b8865e07014fbfed4 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:10:07.951885 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201c84b4-d9af-4048-a89f-3a2cf41d94a7 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:07.952785 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201c84b4-d9af-4048-a89f-3a2cf41d94a7 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:07.953334 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201c84b4-d9af-4048-a89f-3a2cf41d94a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:07.970419 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:10:07.977907 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-201c84b4-d9af-4048-a89f-3a2cf41d94a7 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:10:07.979874 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-201c84b4-d9af-4048-a89f-3a2cf41d94a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:08.072085 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1359/2721] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:10:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:10:08.073734 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72811b28-76ad-4693-a458-61db5de2a9c2 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:08.074125 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72811b28-76ad-4693-a458-61db5de2a9c2 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:08.074614 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72811b28-76ad-4693-a458-61db5de2a9c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:08.097548 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:10:08.105095 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-72811b28-76ad-4693-a458-61db5de2a9c2 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:10:08.105095 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-72811b28-76ad-4693-a458-61db5de2a9c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:08.113006 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1363/2722] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:10:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:10:08.117316 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201c84b4-d9af-4048-a89f-3a2cf41d94a7 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:08.127787 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201c84b4-d9af-4048-a89f-3a2cf41d94a7 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:08.132858 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b134775f-728f-449d-ab9a-519042dbc2d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:08.133671 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b134775f-728f-449d-ab9a-519042dbc2d3 admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:08.133952 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b134775f-728f-449d-ab9a-519042dbc2d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:08.134226 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b134775f-728f-449d-ab9a-519042dbc2d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:08.135546 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b134775f-728f-449d-ab9a-519042dbc2d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:10:08.135852 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b134775f-728f-449d-ab9a-519042dbc2d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:08.138273 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b134775f-728f-449d-ab9a-519042dbc2d3 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:08.155311 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1360/2723] 10.222.0.55 () {58 vars in 1153 bytes} [Wed Aug 24 09:10:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:10:08.157377 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72811b28-76ad-4693-a458-61db5de2a9c2 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:08.172867 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72811b28-76ad-4693-a458-61db5de2a9c2 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:08.180893 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5466a1fa-80a0-4ded-bebc-d493988a6966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:08.182027 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5466a1fa-80a0-4ded-bebc-d493988a6966 admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:08.182247 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5466a1fa-80a0-4ded-bebc-d493988a6966 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:08.182620 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5466a1fa-80a0-4ded-bebc-d493988a6966 admin admin] PATH_INFO: `/v3/projects` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:08.183501 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5466a1fa-80a0-4ded-bebc-d493988a6966 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:08.185611 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5466a1fa-80a0-4ded-bebc-d493988a6966 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:08.386797 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1364/2724] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 09:10:08 2022] POST /identity/v3/projects => generated 369 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:10:08.388710 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b134775f-728f-449d-ab9a-519042dbc2d3 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:08.398245 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b134775f-728f-449d-ab9a-519042dbc2d3 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:08.468381 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bda419-b1c1-43f0-a9f3-de4353df91f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:08.469309 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bda419-b1c1-43f0-a9f3-de4353df91f8 admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:08.469593 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bda419-b1c1-43f0-a9f3-de4353df91f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:08.469852 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bda419-b1c1-43f0-a9f3-de4353df91f8 admin admin] PATH_INFO: `/v3/users` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:08.470493 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bda419-b1c1-43f0-a9f3-de4353df91f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:08.472528 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bda419-b1c1-43f0-a9f3-de4353df91f8 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:08.592357 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1361/2725] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 09:10:08 2022] POST /identity/v3/users => generated 285 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:10:08.597657 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5466a1fa-80a0-4ded-bebc-d493988a6966 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:08.610556 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5466a1fa-80a0-4ded-bebc-d493988a6966 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:08.617238 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb16a98-7712-4996-b3bd-caf29fc72bb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:08.617238 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb16a98-7712-4996-b3bd-caf29fc72bb3 admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:08.617238 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb16a98-7712-4996-b3bd-caf29fc72bb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:08.617238 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb16a98-7712-4996-b3bd-caf29fc72bb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:08.617986 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb16a98-7712-4996-b3bd-caf29fc72bb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:08.619124 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb16a98-7712-4996-b3bd-caf29fc72bb3 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:08.630878 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1365/2726] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:10:08 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:10:08.634956 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bda419-b1c1-43f0-a9f3-de4353df91f8 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:08.645681 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bda419-b1c1-43f0-a9f3-de4353df91f8 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:08.650200 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378876cf-5643-4cb1-b0db-199544a16755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:08.650813 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378876cf-5643-4cb1-b0db-199544a16755 admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:08.651088 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378876cf-5643-4cb1-b0db-199544a16755 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:08.651361 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378876cf-5643-4cb1-b0db-199544a16755 admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:08.651989 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378876cf-5643-4cb1-b0db-199544a16755 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:08.653844 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378876cf-5643-4cb1-b0db-199544a16755 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:08.667936 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1362/2727] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:10:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:10:08.672043 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb16a98-7712-4996-b3bd-caf29fc72bb3 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:08.682788 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb16a98-7712-4996-b3bd-caf29fc72bb3 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:08.687413 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b8b075-a38c-439e-86b6-ca12366ce5cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:08.688073 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b8b075-a38c-439e-86b6-ca12366ce5cd admin admin] REQUEST_METHOD: `PUT` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:08.688351 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b8b075-a38c-439e-86b6-ca12366ce5cd admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:08.688669 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b8b075-a38c-439e-86b6-ca12366ce5cd admin admin] PATH_INFO: `/v3/projects/e63591d806bd4e4a92a5541a9b961ab7/users/e7334516a54748358f4f23926d718860/roles/b674fd0a592e4e0cada811bba5ceea4d` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:08.705427 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b8b075-a38c-439e-86b6-ca12366ce5cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=e63591d806bd4e4a92a5541a9b961ab7, user_id=e7334516a54748358f4f23926d718860, role_id=b674fd0a592e4e0cada811bba5ceea4d)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:08.708127 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b8b075-a38c-439e-86b6-ca12366ce5cd admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:08.762018 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1366/2728] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 09:10:08 2022] PUT /identity/v3/projects/e63591d806bd4e4a92a5541a9b961ab7/users/e7334516a54748358f4f23926d718860/roles/b674fd0a592e4e0cada811bba5ceea4d => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:10:08.773624 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb274bf7-ebeb-4790-bcd3-6d27ac184280 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:08.773929 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb274bf7-ebeb-4790-bcd3-6d27ac184280 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:08.774200 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb274bf7-ebeb-4790-bcd3-6d27ac184280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:08.802405 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-bb274bf7-ebeb-4790-bcd3-6d27ac184280 None None] MFA Rules not processed for user `e7334516a54748358f4f23926d718860`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:10:08.803535 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-bb274bf7-ebeb-4790-bcd3-6d27ac184280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:08.871008 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1363/2729] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:10:08 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:10:08.875531 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b8b075-a38c-439e-86b6-ca12366ce5cd admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:08.941071 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b8b075-a38c-439e-86b6-ca12366ce5cd admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:08.946062 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d3e0bf-e28c-4a3f-8cb1-00a22025ec75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:08.946679 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d3e0bf-e28c-4a3f-8cb1-00a22025ec75 admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:08.946957 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d3e0bf-e28c-4a3f-8cb1-00a22025ec75 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:08.947275 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d3e0bf-e28c-4a3f-8cb1-00a22025ec75 admin admin] PATH_INFO: `/v3/projects` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:08.947931 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d3e0bf-e28c-4a3f-8cb1-00a22025ec75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:08.949838 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d3e0bf-e28c-4a3f-8cb1-00a22025ec75 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:09.034486 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1367/2730] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 09:10:08 2022] POST /identity/v3/projects => generated 367 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:10:09.034778 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb274bf7-ebeb-4790-bcd3-6d27ac184280 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:09.040863 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:10:09.048267 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb274bf7-ebeb-4790-bcd3-6d27ac184280 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:09.110298 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe30fc6-f027-468a-8265-1e0f070e4df4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:09.110996 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe30fc6-f027-468a-8265-1e0f070e4df4 admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:09.111277 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe30fc6-f027-468a-8265-1e0f070e4df4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:09.111555 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe30fc6-f027-468a-8265-1e0f070e4df4 admin admin] PATH_INFO: `/v3/users` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:09.112247 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe30fc6-f027-468a-8265-1e0f070e4df4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:09.114257 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe30fc6-f027-468a-8265-1e0f070e4df4 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:09.177938 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1364/2731] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 09:10:09 2022] POST /identity/v3/users => generated 290 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:10:09.185182 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d3e0bf-e28c-4a3f-8cb1-00a22025ec75 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:09.195607 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d3e0bf-e28c-4a3f-8cb1-00a22025ec75 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:09.200300 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b464d5f0-903a-46c0-af41-b128ece4cc4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:09.200946 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b464d5f0-903a-46c0-af41-b128ece4cc4b admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:09.201233 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b464d5f0-903a-46c0-af41-b128ece4cc4b admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:09.201505 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b464d5f0-903a-46c0-af41-b128ece4cc4b admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:09.202148 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b464d5f0-903a-46c0-af41-b128ece4cc4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:09.203744 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b464d5f0-903a-46c0-af41-b128ece4cc4b admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:09.215444 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1368/2732] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:10:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:10:09.219644 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe30fc6-f027-468a-8265-1e0f070e4df4 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:09.230207 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe30fc6-f027-468a-8265-1e0f070e4df4 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:09.234877 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0c4896-e14f-4897-bab9-d2b96db856c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:09.235514 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0c4896-e14f-4897-bab9-d2b96db856c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:09.235792 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0c4896-e14f-4897-bab9-d2b96db856c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:09.236065 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0c4896-e14f-4897-bab9-d2b96db856c1 admin admin] PATH_INFO: `/v3/projects/00d0c6fb47b9400e95bc5efc4f0eab2b/users/d4f2e1eea6354715928020f3585a33b3/roles/b870cd1acaa744e1ae1d58b9ac283821` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:09.250434 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0c4896-e14f-4897-bab9-d2b96db856c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00d0c6fb47b9400e95bc5efc4f0eab2b, user_id=d4f2e1eea6354715928020f3585a33b3, role_id=b870cd1acaa744e1ae1d58b9ac283821)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:09.253123 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0c4896-e14f-4897-bab9-d2b96db856c1 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:09.338679 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1365/2733] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 09:10:09 2022] PUT /identity/v3/projects/00d0c6fb47b9400e95bc5efc4f0eab2b/users/d4f2e1eea6354715928020f3585a33b3/roles/b870cd1acaa744e1ae1d58b9ac283821 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:10:09.342547 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b464d5f0-903a-46c0-af41-b128ece4cc4b admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:09.352937 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b464d5f0-903a-46c0-af41-b128ece4cc4b admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:09.412398 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cedc71-dfb5-429b-8af1-18a2e52d8f9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:09.413034 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cedc71-dfb5-429b-8af1-18a2e52d8f9f admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:09.413316 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cedc71-dfb5-429b-8af1-18a2e52d8f9f admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:09.413589 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cedc71-dfb5-429b-8af1-18a2e52d8f9f admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:09.414226 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cedc71-dfb5-429b-8af1-18a2e52d8f9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:09.415799 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cedc71-dfb5-429b-8af1-18a2e52d8f9f admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:09.426168 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1369/2734] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:10:09 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:10:09.430372 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0c4896-e14f-4897-bab9-d2b96db856c1 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:09.440985 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0c4896-e14f-4897-bab9-d2b96db856c1 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:09.445544 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b4dead-e1fe-416e-b969-389852255a24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:09.446173 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b4dead-e1fe-416e-b969-389852255a24 admin admin] REQUEST_METHOD: `PUT` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:09.446443 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b4dead-e1fe-416e-b969-389852255a24 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:09.446707 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b4dead-e1fe-416e-b969-389852255a24 admin admin] PATH_INFO: `/v3/projects/00d0c6fb47b9400e95bc5efc4f0eab2b/users/d4f2e1eea6354715928020f3585a33b3/roles/b870cd1acaa744e1ae1d58b9ac283821` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:09.449242 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b4dead-e1fe-416e-b969-389852255a24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00d0c6fb47b9400e95bc5efc4f0eab2b, user_id=d4f2e1eea6354715928020f3585a33b3, role_id=b870cd1acaa744e1ae1d58b9ac283821)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:09.451852 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b4dead-e1fe-416e-b969-389852255a24 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:09.462695 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1366/2735] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 09:10:09 2022] PUT /identity/v3/projects/00d0c6fb47b9400e95bc5efc4f0eab2b/users/d4f2e1eea6354715928020f3585a33b3/roles/b870cd1acaa744e1ae1d58b9ac283821 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:10:09.472459 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7dab80-6a49-4645-8fc6-b6e2e634d313 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:09.472778 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7dab80-6a49-4645-8fc6-b6e2e634d313 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:09.473052 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7dab80-6a49-4645-8fc6-b6e2e634d313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:09.501064 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:10:09.505945 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-ce7dab80-6a49-4645-8fc6-b6e2e634d313 None None] MFA Rules not processed for user `d4f2e1eea6354715928020f3585a33b3`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:10:09.507283 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-ce7dab80-6a49-4645-8fc6-b6e2e634d313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:09.585256 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1370/2736] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:10:09 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:10:09.596965 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b4dead-e1fe-416e-b969-389852255a24 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:09.599110 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-75b4dead-e1fe-416e-b969-389852255a24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:09.667612 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b4dead-e1fe-416e-b969-389852255a24 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:09.672493 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cc6ca7-8439-49fa-93aa-19bf6cc30de7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbb9ac667f04c35a63b2689df18efed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:09.673200 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cc6ca7-8439-49fa-93aa-19bf6cc30de7 service cinder] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:09.673543 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cc6ca7-8439-49fa-93aa-19bf6cc30de7 service cinder] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:09.673880 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cc6ca7-8439-49fa-93aa-19bf6cc30de7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:09.684016 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cc6ca7-8439-49fa-93aa-19bf6cc30de7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:09.685465 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cc6ca7-8439-49fa-93aa-19bf6cc30de7 service cinder] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:09.695185 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1367/2737] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:10:09 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:10:12.600288 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7dab80-6a49-4645-8fc6-b6e2e634d313 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:12.661878 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7dab80-6a49-4645-8fc6-b6e2e634d313 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:12.667231 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2267a04a-425a-452d-b4ea-2414f621c06d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:12.667841 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2267a04a-425a-452d-b4ea-2414f621c06d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:12.668119 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2267a04a-425a-452d-b4ea-2414f621c06d admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:12.668389 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2267a04a-425a-452d-b4ea-2414f621c06d admin admin] PATH_INFO: `/v3/users/e7334516a54748358f4f23926d718860` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:12.669512 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2267a04a-425a-452d-b4ea-2414f621c06d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7334516a54748358f4f23926d718860)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:12.671423 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2267a04a-425a-452d-b4ea-2414f621c06d admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:12.853434 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-2267a04a-425a-452d-b4ea-2414f621c06d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7334516a54748358f4f23926d718860'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:12.854562 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-2267a04a-425a-452d-b4ea-2414f621c06d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7334516a54748358f4f23926d718860'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:12.855154 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-2267a04a-425a-452d-b4ea-2414f621c06d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7334516a54748358f4f23926d718860'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:12.867017 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-2267a04a-425a-452d-b4ea-2414f621c06d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7334516a54748358f4f23926d718860'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:12.954790 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-2267a04a-425a-452d-b4ea-2414f621c06d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7334516a54748358f4f23926d718860'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:12.968573 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1371/2738] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 09:10:12 2022] DELETE /identity/v3/users/e7334516a54748358f4f23926d718860 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:10:12.974132 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5943e91-2268-46e6-9f6e-2984f95fc9bb None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:12.974596 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5943e91-2268-46e6-9f6e-2984f95fc9bb None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:12.974703 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5943e91-2268-46e6-9f6e-2984f95fc9bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:12.994906 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-f5943e91-2268-46e6-9f6e-2984f95fc9bb None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:10:13.003653 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-f5943e91-2268-46e6-9f6e-2984f95fc9bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:13.049683 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:10:13.061601 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1368/2739] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:10:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:10:13.071398 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2267a04a-425a-452d-b4ea-2414f621c06d admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:13.072882 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-2267a04a-425a-452d-b4ea-2414f621c06d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:13.094086 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:10:13.134209 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2267a04a-425a-452d-b4ea-2414f621c06d admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:13.138455 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2051bb34-a779-435a-a210-463bcd872e8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:13.139132 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2051bb34-a779-435a-a210-463bcd872e8b service neutron] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:13.139413 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2051bb34-a779-435a-a210-463bcd872e8b service neutron] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:13.139687 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2051bb34-a779-435a-a210-463bcd872e8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:13.149433 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2051bb34-a779-435a-a210-463bcd872e8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:13.151478 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2051bb34-a779-435a-a210-463bcd872e8b service neutron] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:13.161622 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1372/2740] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:10:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:10:13.580566 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5943e91-2268-46e6-9f6e-2984f95fc9bb None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:13.582034 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-f5943e91-2268-46e6-9f6e-2984f95fc9bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:13.594896 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5943e91-2268-46e6-9f6e-2984f95fc9bb None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:13.599493 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d057d6-3fab-42bc-afa3-9dfa0c062209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:13.603249 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d057d6-3fab-42bc-afa3-9dfa0c062209 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:13.603249 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d057d6-3fab-42bc-afa3-9dfa0c062209 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:13.603249 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d057d6-3fab-42bc-afa3-9dfa0c062209 admin admin] PATH_INFO: `/v3/projects/e63591d806bd4e4a92a5541a9b961ab7` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:13.603249 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d057d6-3fab-42bc-afa3-9dfa0c062209 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e63591d806bd4e4a92a5541a9b961ab7)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:13.603931 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d057d6-3fab-42bc-afa3-9dfa0c062209 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:13.715950 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-e7d057d6-3fab-42bc-afa3-9dfa0c062209 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e63591d806bd4e4a92a5541a9b961ab7', 'request_id': 'req-e7d057d6-3fab-42bc-afa3-9dfa0c062209'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:13.794692 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-e7d057d6-3fab-42bc-afa3-9dfa0c062209 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e63591d806bd4e4a92a5541a9b961ab7', 'request_id': 'req-e7d057d6-3fab-42bc-afa3-9dfa0c062209'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:13.801634 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-e7d057d6-3fab-42bc-afa3-9dfa0c062209 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e63591d806bd4e4a92a5541a9b961ab7', 'request_id': 'req-e7d057d6-3fab-42bc-afa3-9dfa0c062209'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:13.812830 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-e7d057d6-3fab-42bc-afa3-9dfa0c062209 admin admin] The token cache is being invalidate because project e63591d806bd4e4a92a5541a9b961ab7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:10:13.813118 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-e7d057d6-3fab-42bc-afa3-9dfa0c062209 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:13.813765 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-e7d057d6-3fab-42bc-afa3-9dfa0c062209 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:13.815258 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1369/2741] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 09:10:13 2022] DELETE /identity/v3/projects/e63591d806bd4e4a92a5541a9b961ab7 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:10:13.819577 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2051bb34-a779-435a-a210-463bcd872e8b service neutron] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:13.830441 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2051bb34-a779-435a-a210-463bcd872e8b service neutron] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:13.889367 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72fe108-4cdd-4653-a207-e825543f1522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:13.890006 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72fe108-4cdd-4653-a207-e825543f1522 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:13.890353 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72fe108-4cdd-4653-a207-e825543f1522 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:13.890626 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72fe108-4cdd-4653-a207-e825543f1522 admin admin] PATH_INFO: `/v3/users/d4f2e1eea6354715928020f3585a33b3` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:13.891760 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72fe108-4cdd-4653-a207-e825543f1522 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4f2e1eea6354715928020f3585a33b3)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:13.893723 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72fe108-4cdd-4653-a207-e825543f1522 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:14.217890 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-f72fe108-4cdd-4653-a207-e825543f1522 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4f2e1eea6354715928020f3585a33b3'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:14.218935 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-f72fe108-4cdd-4653-a207-e825543f1522 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4f2e1eea6354715928020f3585a33b3'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:14.219625 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-f72fe108-4cdd-4653-a207-e825543f1522 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4f2e1eea6354715928020f3585a33b3'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:14.233786 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-f72fe108-4cdd-4653-a207-e825543f1522 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4f2e1eea6354715928020f3585a33b3'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:14.333132 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-f72fe108-4cdd-4653-a207-e825543f1522 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4f2e1eea6354715928020f3585a33b3'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:14.354929 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1373/2742] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 09:10:13 2022] DELETE /identity/v3/users/d4f2e1eea6354715928020f3585a33b3 => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:10:14.796817 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d057d6-3fab-42bc-afa3-9dfa0c062209 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:14.798463 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-e7d057d6-3fab-42bc-afa3-9dfa0c062209 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:14.820143 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d057d6-3fab-42bc-afa3-9dfa0c062209 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:14.893697 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6e1af7-8cd8-44ce-8868-94a4dee05e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:14.894389 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6e1af7-8cd8-44ce-8868-94a4dee05e1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:14.895806 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6e1af7-8cd8-44ce-8868-94a4dee05e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:14.895806 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6e1af7-8cd8-44ce-8868-94a4dee05e1a admin admin] PATH_INFO: `/v3/projects/00d0c6fb47b9400e95bc5efc4f0eab2b` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:14.897928 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6e1af7-8cd8-44ce-8868-94a4dee05e1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=00d0c6fb47b9400e95bc5efc4f0eab2b)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:14.899127 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6e1af7-8cd8-44ce-8868-94a4dee05e1a admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:15.019437 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-9b6e1af7-8cd8-44ce-8868-94a4dee05e1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00d0c6fb47b9400e95bc5efc4f0eab2b', 'request_id': 'req-9b6e1af7-8cd8-44ce-8868-94a4dee05e1a'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:15.119794 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-9b6e1af7-8cd8-44ce-8868-94a4dee05e1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00d0c6fb47b9400e95bc5efc4f0eab2b', 'request_id': 'req-9b6e1af7-8cd8-44ce-8868-94a4dee05e1a'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:15.129374 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-9b6e1af7-8cd8-44ce-8868-94a4dee05e1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00d0c6fb47b9400e95bc5efc4f0eab2b', 'request_id': 'req-9b6e1af7-8cd8-44ce-8868-94a4dee05e1a'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:15.142120 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-9b6e1af7-8cd8-44ce-8868-94a4dee05e1a admin admin] The token cache is being invalidate because project 00d0c6fb47b9400e95bc5efc4f0eab2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:10:15.142511 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-9b6e1af7-8cd8-44ce-8868-94a4dee05e1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:15.143311 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-9b6e1af7-8cd8-44ce-8868-94a4dee05e1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:15.144951 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1370/2743] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 09:10:14 2022] DELETE /identity/v3/projects/00d0c6fb47b9400e95bc5efc4f0eab2b => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:10:15.149996 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3554a3-8e81-43a7-803d-f89ca1cf1b76 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:15.151805 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3554a3-8e81-43a7-803d-f89ca1cf1b76 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:15.151805 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3554a3-8e81-43a7-803d-f89ca1cf1b76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:15.179901 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-fe3554a3-8e81-43a7-803d-f89ca1cf1b76 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:10:15.181147 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-fe3554a3-8e81-43a7-803d-f89ca1cf1b76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:15.225063 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:10:15.252123 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1374/2744] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:10:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:10:15.261800 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655b1475-4db1-441b-b562-7abb9ed4fec7 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:15.262121 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655b1475-4db1-441b-b562-7abb9ed4fec7 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:15.262396 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655b1475-4db1-441b-b562-7abb9ed4fec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:15.282151 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-655b1475-4db1-441b-b562-7abb9ed4fec7 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:10:15.283432 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-655b1475-4db1-441b-b562-7abb9ed4fec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:15.292890 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1371/2745] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:10:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:10:15.297270 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3554a3-8e81-43a7-803d-f89ca1cf1b76 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:15.308084 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3554a3-8e81-43a7-803d-f89ca1cf1b76 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:15.312980 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd6dfe6-9a77-4ecf-9fc9-704cc8dbaba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:15.313305 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd6dfe6-9a77-4ecf-9fc9-704cc8dbaba7 admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:15.313582 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd6dfe6-9a77-4ecf-9fc9-704cc8dbaba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:15.313851 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd6dfe6-9a77-4ecf-9fc9-704cc8dbaba7 admin admin] PATH_INFO: `/v3/domains` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:15.314524 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd6dfe6-9a77-4ecf-9fc9-704cc8dbaba7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:10:15.314825 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd6dfe6-9a77-4ecf-9fc9-704cc8dbaba7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:15.316411 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd6dfe6-9a77-4ecf-9fc9-704cc8dbaba7 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:15.329753 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1375/2746] 10.222.0.55 () {58 vars in 1153 bytes} [Wed Aug 24 09:10:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:10:15.334267 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655b1475-4db1-441b-b562-7abb9ed4fec7 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:15.339914 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:10:15.343706 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655b1475-4db1-441b-b562-7abb9ed4fec7 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:15.348461 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ebc53f-326d-4f96-944e-a31a61bd289e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:15.349081 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ebc53f-326d-4f96-944e-a31a61bd289e admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:15.349355 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ebc53f-326d-4f96-944e-a31a61bd289e admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:15.349622 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ebc53f-326d-4f96-944e-a31a61bd289e admin admin] PATH_INFO: `/v3/projects` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:15.350243 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ebc53f-326d-4f96-944e-a31a61bd289e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:15.352124 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ebc53f-326d-4f96-944e-a31a61bd289e admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:15.439210 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1372/2747] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 09:10:15 2022] POST /identity/v3/projects => generated 375 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:10:15.443367 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd6dfe6-9a77-4ecf-9fc9-704cc8dbaba7 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:15.455273 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd6dfe6-9a77-4ecf-9fc9-704cc8dbaba7 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:15.522053 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cd92a0-846c-459a-8fcf-a00fb76ccc13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:15.522669 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cd92a0-846c-459a-8fcf-a00fb76ccc13 admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:15.522965 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cd92a0-846c-459a-8fcf-a00fb76ccc13 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:15.523242 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cd92a0-846c-459a-8fcf-a00fb76ccc13 admin admin] PATH_INFO: `/v3/users` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:15.523885 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cd92a0-846c-459a-8fcf-a00fb76ccc13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:15.525897 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cd92a0-846c-459a-8fcf-a00fb76ccc13 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:15.657638 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1376/2748] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 09:10:15 2022] POST /identity/v3/users => generated 288 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:10:15.661734 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ebc53f-326d-4f96-944e-a31a61bd289e admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:15.672646 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ebc53f-326d-4f96-944e-a31a61bd289e admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:15.677235 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88ad9f1-40f8-470f-aaae-ae83cef23a21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:15.677866 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88ad9f1-40f8-470f-aaae-ae83cef23a21 admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:15.678150 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88ad9f1-40f8-470f-aaae-ae83cef23a21 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:15.678420 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88ad9f1-40f8-470f-aaae-ae83cef23a21 admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:15.679049 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88ad9f1-40f8-470f-aaae-ae83cef23a21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:15.680659 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88ad9f1-40f8-470f-aaae-ae83cef23a21 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:15.691057 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1373/2749] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:10:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:10:15.695277 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cd92a0-846c-459a-8fcf-a00fb76ccc13 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:15.705646 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cd92a0-846c-459a-8fcf-a00fb76ccc13 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:15.710700 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bf73ba-093c-4ffc-98ed-0e681454f0c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:15.711358 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bf73ba-093c-4ffc-98ed-0e681454f0c3 admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:15.711697 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bf73ba-093c-4ffc-98ed-0e681454f0c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:15.711983 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bf73ba-093c-4ffc-98ed-0e681454f0c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:15.712692 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bf73ba-093c-4ffc-98ed-0e681454f0c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:15.714307 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bf73ba-093c-4ffc-98ed-0e681454f0c3 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:15.725356 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1377/2750] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:10:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:10:15.732436 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88ad9f1-40f8-470f-aaae-ae83cef23a21 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:15.742649 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88ad9f1-40f8-470f-aaae-ae83cef23a21 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:15.747874 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d6b031-2aea-440f-b3c8-61fcdbf7b858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:15.752278 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d6b031-2aea-440f-b3c8-61fcdbf7b858 admin admin] REQUEST_METHOD: `PUT` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:15.752278 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d6b031-2aea-440f-b3c8-61fcdbf7b858 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:15.752278 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d6b031-2aea-440f-b3c8-61fcdbf7b858 admin admin] PATH_INFO: `/v3/projects/9fadc8cbdc53422db8e82919ac88d4b5/users/ced09713301e43988e7346203fdf0a9e/roles/b674fd0a592e4e0cada811bba5ceea4d` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:15.766917 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d6b031-2aea-440f-b3c8-61fcdbf7b858 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fadc8cbdc53422db8e82919ac88d4b5, user_id=ced09713301e43988e7346203fdf0a9e, role_id=b674fd0a592e4e0cada811bba5ceea4d)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:15.772323 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d6b031-2aea-440f-b3c8-61fcdbf7b858 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:15.858412 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1374/2751] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 09:10:15 2022] PUT /identity/v3/projects/9fadc8cbdc53422db8e82919ac88d4b5/users/ced09713301e43988e7346203fdf0a9e/roles/b674fd0a592e4e0cada811bba5ceea4d => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:10:15.927473 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5483b06-56bf-4395-a083-26fe47b6a6d6 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:15.927795 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5483b06-56bf-4395-a083-26fe47b6a6d6 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:15.928078 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5483b06-56bf-4395-a083-26fe47b6a6d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:15.960497 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-e5483b06-56bf-4395-a083-26fe47b6a6d6 None None] MFA Rules not processed for user `ced09713301e43988e7346203fdf0a9e`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:10:15.961920 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-e5483b06-56bf-4395-a083-26fe47b6a6d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:16.034866 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1378/2752] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:10:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:10:16.050600 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d6b031-2aea-440f-b3c8-61fcdbf7b858 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:16.050600 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-c1d6b031-2aea-440f-b3c8-61fcdbf7b858 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:16.108155 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d6b031-2aea-440f-b3c8-61fcdbf7b858 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:16.111889 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc5db08-bf69-4b94-9507-059e54596fb9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbb9ac667f04c35a63b2689df18efed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:16.112496 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc5db08-bf69-4b94-9507-059e54596fb9 service cinder] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:16.112809 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc5db08-bf69-4b94-9507-059e54596fb9 service cinder] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:16.113081 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc5db08-bf69-4b94-9507-059e54596fb9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:16.122287 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc5db08-bf69-4b94-9507-059e54596fb9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:16.123744 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc5db08-bf69-4b94-9507-059e54596fb9 service cinder] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:16.132691 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1375/2753] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:10:16 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:10:16.862365 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5483b06-56bf-4395-a083-26fe47b6a6d6 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:16.862365 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-e5483b06-56bf-4395-a083-26fe47b6a6d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:16.906469 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:10:16.928091 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5483b06-56bf-4395-a083-26fe47b6a6d6 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:16.933141 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6299f7-afc6-48fd-b8b5-5d9d9cbc9453 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:16.933783 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6299f7-afc6-48fd-b8b5-5d9d9cbc9453 service neutron] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:16.934049 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6299f7-afc6-48fd-b8b5-5d9d9cbc9453 service neutron] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:16.934317 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6299f7-afc6-48fd-b8b5-5d9d9cbc9453 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:16.936149 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-cf6299f7-afc6-48fd-b8b5-5d9d9cbc9453 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:17.003797 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6299f7-afc6-48fd-b8b5-5d9d9cbc9453 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:17.005348 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6299f7-afc6-48fd-b8b5-5d9d9cbc9453 service neutron] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:17.017142 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1379/2754] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:10:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:10:18.026554 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc5db08-bf69-4b94-9507-059e54596fb9 service cinder] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:18.028402 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-5cc5db08-bf69-4b94-9507-059e54596fb9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:18.044469 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc5db08-bf69-4b94-9507-059e54596fb9 service cinder] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:18.050685 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec2af5e-51c6-49f0-850d-04f867b6add4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:18.051654 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec2af5e-51c6-49f0-850d-04f867b6add4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:18.052104 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec2af5e-51c6-49f0-850d-04f867b6add4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:18.052593 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec2af5e-51c6-49f0-850d-04f867b6add4 admin admin] PATH_INFO: `/v3/users/8ecfe4a5ab414a9a9b80edb224c82699` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:18.054162 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec2af5e-51c6-49f0-850d-04f867b6add4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ecfe4a5ab414a9a9b80edb224c82699)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:18.056407 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec2af5e-51c6-49f0-850d-04f867b6add4 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:18.318431 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-9ec2af5e-51c6-49f0-850d-04f867b6add4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ecfe4a5ab414a9a9b80edb224c82699'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:18.319510 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-9ec2af5e-51c6-49f0-850d-04f867b6add4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ecfe4a5ab414a9a9b80edb224c82699'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:18.321009 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-9ec2af5e-51c6-49f0-850d-04f867b6add4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ecfe4a5ab414a9a9b80edb224c82699'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:18.343469 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-9ec2af5e-51c6-49f0-850d-04f867b6add4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ecfe4a5ab414a9a9b80edb224c82699'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:18.475408 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-9ec2af5e-51c6-49f0-850d-04f867b6add4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ecfe4a5ab414a9a9b80edb224c82699'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:18.492771 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1376/2755] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 09:10:18 2022] DELETE /identity/v3/users/8ecfe4a5ab414a9a9b80edb224c82699 => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:10:18.499394 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb598dd3-ddbc-475a-b292-ec3bfe40f78a None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:18.499900 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb598dd3-ddbc-475a-b292-ec3bfe40f78a None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:18.500004 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb598dd3-ddbc-475a-b292-ec3bfe40f78a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:18.523588 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-eb598dd3-ddbc-475a-b292-ec3bfe40f78a None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:10:18.524795 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-eb598dd3-ddbc-475a-b292-ec3bfe40f78a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:18.595191 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1380/2756] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:10:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:10:18.606399 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec2af5e-51c6-49f0-850d-04f867b6add4 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:18.608316 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-9ec2af5e-51c6-49f0-850d-04f867b6add4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:18.635845 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:10:18.668048 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec2af5e-51c6-49f0-850d-04f867b6add4 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:18.671805 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae4df3a-ff53-42c1-a8b1-e91dd53ca39c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:18.672414 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae4df3a-ff53-42c1-a8b1-e91dd53ca39c service neutron] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:18.672724 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae4df3a-ff53-42c1-a8b1-e91dd53ca39c service neutron] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:18.673025 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae4df3a-ff53-42c1-a8b1-e91dd53ca39c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:18.683080 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae4df3a-ff53-42c1-a8b1-e91dd53ca39c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:18.684567 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae4df3a-ff53-42c1-a8b1-e91dd53ca39c service neutron] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:18.699465 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1377/2757] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:10:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:10:19.290161 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb598dd3-ddbc-475a-b292-ec3bfe40f78a None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:19.293043 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-eb598dd3-ddbc-475a-b292-ec3bfe40f78a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:19.323324 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb598dd3-ddbc-475a-b292-ec3bfe40f78a None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:19.332930 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91942afd-ded4-4af3-9f36-a139b17b78b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:19.333836 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91942afd-ded4-4af3-9f36-a139b17b78b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:19.334275 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91942afd-ded4-4af3-9f36-a139b17b78b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:19.334721 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91942afd-ded4-4af3-9f36-a139b17b78b1 admin admin] PATH_INFO: `/v3/projects/cf6a786aaf254d6fa32b32721b0b6645` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:19.337712 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91942afd-ded4-4af3-9f36-a139b17b78b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf6a786aaf254d6fa32b32721b0b6645)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:19.340943 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91942afd-ded4-4af3-9f36-a139b17b78b1 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:19.489540 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-91942afd-ded4-4af3-9f36-a139b17b78b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf6a786aaf254d6fa32b32721b0b6645', 'request_id': 'req-91942afd-ded4-4af3-9f36-a139b17b78b1'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:19.496879 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-91942afd-ded4-4af3-9f36-a139b17b78b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf6a786aaf254d6fa32b32721b0b6645', 'request_id': 'req-91942afd-ded4-4af3-9f36-a139b17b78b1'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:19.600679 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-91942afd-ded4-4af3-9f36-a139b17b78b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf6a786aaf254d6fa32b32721b0b6645', 'request_id': 'req-91942afd-ded4-4af3-9f36-a139b17b78b1'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:19.615214 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-91942afd-ded4-4af3-9f36-a139b17b78b1 admin admin] The token cache is being invalidate because project cf6a786aaf254d6fa32b32721b0b6645 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:10:19.615214 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-91942afd-ded4-4af3-9f36-a139b17b78b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:19.615214 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-91942afd-ded4-4af3-9f36-a139b17b78b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:19.615707 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1381/2758] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 09:10:19 2022] DELETE /identity/v3/projects/cf6a786aaf254d6fa32b32721b0b6645 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:10:19.619701 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae4df3a-ff53-42c1-a8b1-e91dd53ca39c service neutron] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:19.629611 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae4df3a-ff53-42c1-a8b1-e91dd53ca39c service neutron] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:19.690510 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4c62fd-1ac7-44a1-a458-7157f9c51e1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:19.691137 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4c62fd-1ac7-44a1-a458-7157f9c51e1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:19.691418 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4c62fd-1ac7-44a1-a458-7157f9c51e1c admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:19.691688 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4c62fd-1ac7-44a1-a458-7157f9c51e1c admin admin] PATH_INFO: `/v3/users/c6bec127606547fcbf4a7b3756de4a9f` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:19.692841 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4c62fd-1ac7-44a1-a458-7157f9c51e1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6bec127606547fcbf4a7b3756de4a9f)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:19.694749 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4c62fd-1ac7-44a1-a458-7157f9c51e1c admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:19.941926 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-7d4c62fd-1ac7-44a1-a458-7157f9c51e1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6bec127606547fcbf4a7b3756de4a9f'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:19.942698 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-7d4c62fd-1ac7-44a1-a458-7157f9c51e1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6bec127606547fcbf4a7b3756de4a9f'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:19.943641 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-7d4c62fd-1ac7-44a1-a458-7157f9c51e1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6bec127606547fcbf4a7b3756de4a9f'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:19.959668 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-7d4c62fd-1ac7-44a1-a458-7157f9c51e1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6bec127606547fcbf4a7b3756de4a9f'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:20.074776 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-7d4c62fd-1ac7-44a1-a458-7157f9c51e1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6bec127606547fcbf4a7b3756de4a9f'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:20.099027 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1378/2759] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 09:10:19 2022] DELETE /identity/v3/users/c6bec127606547fcbf4a7b3756de4a9f => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:10:20.445413 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91942afd-ded4-4af3-9f36-a139b17b78b1 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:20.447065 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-91942afd-ded4-4af3-9f36-a139b17b78b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:20.462056 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91942afd-ded4-4af3-9f36-a139b17b78b1 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:20.469561 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:10:20.557731 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5b74b7-b965-44e4-bdc3-5c4c148fd5f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:20.559120 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5b74b7-b965-44e4-bdc3-5c4c148fd5f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:20.559630 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5b74b7-b965-44e4-bdc3-5c4c148fd5f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:20.560209 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5b74b7-b965-44e4-bdc3-5c4c148fd5f5 admin admin] PATH_INFO: `/v3/users/ced09713301e43988e7346203fdf0a9e` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:20.561930 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5b74b7-b965-44e4-bdc3-5c4c148fd5f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ced09713301e43988e7346203fdf0a9e)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:20.564431 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5b74b7-b965-44e4-bdc3-5c4c148fd5f5 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:20.684936 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4c62fd-1ac7-44a1-a458-7157f9c51e1c admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:20.686112 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-7d4c62fd-1ac7-44a1-a458-7157f9c51e1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:20.699822 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4c62fd-1ac7-44a1-a458-7157f9c51e1c admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:20.706034 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b955305-0740-453d-95b7-f0cc980303f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:20.710760 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b955305-0740-453d-95b7-f0cc980303f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:20.710760 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b955305-0740-453d-95b7-f0cc980303f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:20.710760 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b955305-0740-453d-95b7-f0cc980303f8 admin admin] PATH_INFO: `/v3/projects/c2655a5059b84131ba58bc83e2993036` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:20.710760 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b955305-0740-453d-95b7-f0cc980303f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2655a5059b84131ba58bc83e2993036)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:20.710760 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b955305-0740-453d-95b7-f0cc980303f8 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:20.792259 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-cd5b74b7-b965-44e4-bdc3-5c4c148fd5f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ced09713301e43988e7346203fdf0a9e'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:20.793011 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-cd5b74b7-b965-44e4-bdc3-5c4c148fd5f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ced09713301e43988e7346203fdf0a9e'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:20.793617 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-cd5b74b7-b965-44e4-bdc3-5c4c148fd5f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ced09713301e43988e7346203fdf0a9e'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:20.808560 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-cd5b74b7-b965-44e4-bdc3-5c4c148fd5f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ced09713301e43988e7346203fdf0a9e'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:20.813430 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:10:20.820389 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-7b955305-0740-453d-95b7-f0cc980303f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2655a5059b84131ba58bc83e2993036', 'request_id': 'req-7b955305-0740-453d-95b7-f0cc980303f8'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:20.892376 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-cd5b74b7-b965-44e4-bdc3-5c4c148fd5f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ced09713301e43988e7346203fdf0a9e'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:20.908266 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1382/2760] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 09:10:20 2022] DELETE /identity/v3/users/ced09713301e43988e7346203fdf0a9e => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:10:20.918386 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5e0710-8e6d-43e3-908c-34470920601f None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:20.918687 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5e0710-8e6d-43e3-908c-34470920601f None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:20.919489 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5e0710-8e6d-43e3-908c-34470920601f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:20.938229 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-7b955305-0740-453d-95b7-f0cc980303f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2655a5059b84131ba58bc83e2993036', 'request_id': 'req-7b955305-0740-453d-95b7-f0cc980303f8'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:20.941703 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-3b5e0710-8e6d-43e3-908c-34470920601f None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:10:20.942952 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-3b5e0710-8e6d-43e3-908c-34470920601f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:20.962232 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-7b955305-0740-453d-95b7-f0cc980303f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2655a5059b84131ba58bc83e2993036', 'request_id': 'req-7b955305-0740-453d-95b7-f0cc980303f8'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:20.982046 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-7b955305-0740-453d-95b7-f0cc980303f8 admin admin] The token cache is being invalidate because project c2655a5059b84131ba58bc83e2993036 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:10:20.982406 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-7b955305-0740-453d-95b7-f0cc980303f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:20.983415 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-7b955305-0740-453d-95b7-f0cc980303f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:20.985266 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1379/2761] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 09:10:20 2022] DELETE /identity/v3/projects/c2655a5059b84131ba58bc83e2993036 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:10:20.990013 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4e8e17-8323-4f23-a1cf-5efe85ca6c67 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:20.990294 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4e8e17-8323-4f23-a1cf-5efe85ca6c67 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:20.990574 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4e8e17-8323-4f23-a1cf-5efe85ca6c67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:21.025751 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-3c4e8e17-8323-4f23-a1cf-5efe85ca6c67 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:10:21.029302 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1383/2762] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:10:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:10:21.029473 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-3c4e8e17-8323-4f23-a1cf-5efe85ca6c67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:21.038103 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5e0710-8e6d-43e3-908c-34470920601f None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:21.043034 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-3b5e0710-8e6d-43e3-908c-34470920601f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:21.056102 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1380/2763] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:10:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:10:21.058689 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0a90e8-ca1e-4d76-9ab2-a3c2a2601dad None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:21.059190 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0a90e8-ca1e-4d76-9ab2-a3c2a2601dad None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:21.059585 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0a90e8-ca1e-4d76-9ab2-a3c2a2601dad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:21.081137 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-3b0a90e8-ca1e-4d76-9ab2-a3c2a2601dad None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:10:21.082375 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-3b0a90e8-ca1e-4d76-9ab2-a3c2a2601dad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:21.091406 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1381/2764] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:10:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:10:21.091598 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:10:21.095719 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0a90e8-ca1e-4d76-9ab2-a3c2a2601dad None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:21.105082 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0a90e8-ca1e-4d76-9ab2-a3c2a2601dad None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:21.109840 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fb4051-b18a-4e87-bc05-f0ed5192bf02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:21.110429 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fb4051-b18a-4e87-bc05-f0ed5192bf02 admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:21.110705 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fb4051-b18a-4e87-bc05-f0ed5192bf02 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:21.110975 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fb4051-b18a-4e87-bc05-f0ed5192bf02 admin admin] PATH_INFO: `/v3/domains` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:21.111617 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fb4051-b18a-4e87-bc05-f0ed5192bf02 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:10:21.111921 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fb4051-b18a-4e87-bc05-f0ed5192bf02 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:21.113620 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fb4051-b18a-4e87-bc05-f0ed5192bf02 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:21.118234 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5e0710-8e6d-43e3-908c-34470920601f None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:21.122561 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b50d57-6751-4e45-a850-bf7f7dbd6548 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:21.123165 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b50d57-6751-4e45-a850-bf7f7dbd6548 service neutron] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:21.123447 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b50d57-6751-4e45-a850-bf7f7dbd6548 service neutron] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:21.123713 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b50d57-6751-4e45-a850-bf7f7dbd6548 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:21.130796 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1382/2765] 10.222.0.55 () {58 vars in 1153 bytes} [Wed Aug 24 09:10:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:10:21.132461 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fb4051-b18a-4e87-bc05-f0ed5192bf02 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:21.134120 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b50d57-6751-4e45-a850-bf7f7dbd6548 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:21.135689 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b50d57-6751-4e45-a850-bf7f7dbd6548 service neutron] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:21.142557 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fb4051-b18a-4e87-bc05-f0ed5192bf02 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:21.145765 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1384/2766] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:10:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:10:21.147655 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dcdeac-34fa-4ad6-921d-99461c71bbaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:21.148255 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dcdeac-34fa-4ad6-921d-99461c71bbaa admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:21.148531 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dcdeac-34fa-4ad6-921d-99461c71bbaa admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:21.148831 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dcdeac-34fa-4ad6-921d-99461c71bbaa admin admin] PATH_INFO: `/v3/projects` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:21.152577 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dcdeac-34fa-4ad6-921d-99461c71bbaa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:21.152577 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dcdeac-34fa-4ad6-921d-99461c71bbaa admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:21.242634 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1383/2767] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 09:10:21 2022] POST /identity/v3/projects => generated 383 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:10:21.246739 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b50d57-6751-4e45-a850-bf7f7dbd6548 service neutron] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:21.256704 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b50d57-6751-4e45-a850-bf7f7dbd6548 service neutron] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:21.316816 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae1d735-546d-4719-89a0-f1dda94b7c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:21.317466 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae1d735-546d-4719-89a0-f1dda94b7c81 admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:21.317751 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae1d735-546d-4719-89a0-f1dda94b7c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:21.318028 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae1d735-546d-4719-89a0-f1dda94b7c81 admin admin] PATH_INFO: `/v3/users` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:21.318658 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae1d735-546d-4719-89a0-f1dda94b7c81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:21.320866 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae1d735-546d-4719-89a0-f1dda94b7c81 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:21.392721 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1385/2768] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 09:10:21 2022] POST /identity/v3/users => generated 292 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:10:21.394078 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dcdeac-34fa-4ad6-921d-99461c71bbaa admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:21.406120 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dcdeac-34fa-4ad6-921d-99461c71bbaa admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:21.410848 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc56312e-04dd-482a-aee8-a89c6e633e3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:21.411523 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc56312e-04dd-482a-aee8-a89c6e633e3c admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:21.411729 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc56312e-04dd-482a-aee8-a89c6e633e3c admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:21.412002 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc56312e-04dd-482a-aee8-a89c6e633e3c admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:21.413007 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc56312e-04dd-482a-aee8-a89c6e633e3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:21.414607 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc56312e-04dd-482a-aee8-a89c6e633e3c admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:21.429291 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1384/2769] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:10:21 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:10:21.434078 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae1d735-546d-4719-89a0-f1dda94b7c81 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:21.444830 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae1d735-546d-4719-89a0-f1dda94b7c81 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:21.451031 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626a24b8-c371-468c-b2ee-c0fe26cb525c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:21.451984 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626a24b8-c371-468c-b2ee-c0fe26cb525c admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:21.452528 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626a24b8-c371-468c-b2ee-c0fe26cb525c admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:21.453308 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626a24b8-c371-468c-b2ee-c0fe26cb525c admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:21.454145 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626a24b8-c371-468c-b2ee-c0fe26cb525c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:21.456245 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626a24b8-c371-468c-b2ee-c0fe26cb525c admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:21.471408 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc56312e-04dd-482a-aee8-a89c6e633e3c admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:21.472351 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1386/2770] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:10:21 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:10:21.482695 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc56312e-04dd-482a-aee8-a89c6e633e3c admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:21.487665 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f44b8a4-7d8a-4f0a-98e8-c505797d9bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:21.488334 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f44b8a4-7d8a-4f0a-98e8-c505797d9bc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:21.489040 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f44b8a4-7d8a-4f0a-98e8-c505797d9bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:21.489040 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f44b8a4-7d8a-4f0a-98e8-c505797d9bc4 admin admin] PATH_INFO: `/v3/projects/bf658344bfc44fed9bb250573ed1cb28/users/0fb57b7fb7f34202ae37af788309736a/roles/b674fd0a592e4e0cada811bba5ceea4d` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:21.507422 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f44b8a4-7d8a-4f0a-98e8-c505797d9bc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf658344bfc44fed9bb250573ed1cb28, user_id=0fb57b7fb7f34202ae37af788309736a, role_id=b674fd0a592e4e0cada811bba5ceea4d)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:21.510185 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f44b8a4-7d8a-4f0a-98e8-c505797d9bc4 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:21.575637 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1385/2771] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 09:10:21 2022] PUT /identity/v3/projects/bf658344bfc44fed9bb250573ed1cb28/users/0fb57b7fb7f34202ae37af788309736a/roles/b674fd0a592e4e0cada811bba5ceea4d => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:10:21.586291 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfc4a52-8a7a-49e8-a4a8-8cf4a375e526 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:21.586757 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfc4a52-8a7a-49e8-a4a8-8cf4a375e526 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:21.587231 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfc4a52-8a7a-49e8-a4a8-8cf4a375e526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:21.620497 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-6bfc4a52-8a7a-49e8-a4a8-8cf4a375e526 None None] MFA Rules not processed for user `0fb57b7fb7f34202ae37af788309736a`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:10:21.621859 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-6bfc4a52-8a7a-49e8-a4a8-8cf4a375e526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:21.716346 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1387/2772] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:10:21 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:10:21.721047 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f44b8a4-7d8a-4f0a-98e8-c505797d9bc4 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:21.769248 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfc4a52-8a7a-49e8-a4a8-8cf4a375e526 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:21.771495 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-6bfc4a52-8a7a-49e8-a4a8-8cf4a375e526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:21.787738 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f44b8a4-7d8a-4f0a-98e8-c505797d9bc4 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:21.793245 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf1c526-024c-4780-be5f-8e3cf3e1c1db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:21.794177 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf1c526-024c-4780-be5f-8e3cf3e1c1db admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:21.794655 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf1c526-024c-4780-be5f-8e3cf3e1c1db admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:21.795124 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf1c526-024c-4780-be5f-8e3cf3e1c1db admin admin] PATH_INFO: `/v3/projects` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:21.795981 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf1c526-024c-4780-be5f-8e3cf3e1c1db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:21.798165 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf1c526-024c-4780-be5f-8e3cf3e1c1db admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:21.828795 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:10:21.851674 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfc4a52-8a7a-49e8-a4a8-8cf4a375e526 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:21.856717 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858ec981-89a6-403c-8aef-129a2722cc0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:21.857363 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858ec981-89a6-403c-8aef-129a2722cc0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:21.857643 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858ec981-89a6-403c-8aef-129a2722cc0c admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:21.857918 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858ec981-89a6-403c-8aef-129a2722cc0c admin admin] PATH_INFO: `/v3/projects/9fadc8cbdc53422db8e82919ac88d4b5` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:21.859053 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858ec981-89a6-403c-8aef-129a2722cc0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9fadc8cbdc53422db8e82919ac88d4b5)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:21.861127 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858ec981-89a6-403c-8aef-129a2722cc0c admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:21.925629 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1386/2773] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 09:10:21 2022] POST /identity/v3/projects => generated 383 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:10:21.926692 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf1c526-024c-4780-be5f-8e3cf3e1c1db admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:21.963653 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:10:21.965516 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-858ec981-89a6-403c-8aef-129a2722cc0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9fadc8cbdc53422db8e82919ac88d4b5', 'request_id': 'req-858ec981-89a6-403c-8aef-129a2722cc0c'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:21.972110 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-858ec981-89a6-403c-8aef-129a2722cc0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9fadc8cbdc53422db8e82919ac88d4b5', 'request_id': 'req-858ec981-89a6-403c-8aef-129a2722cc0c'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:21.998333 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf1c526-024c-4780-be5f-8e3cf3e1c1db admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:22.054297 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a074479-9425-4d41-9ce6-dabd9b079d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:22.054902 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a074479-9425-4d41-9ce6-dabd9b079d18 admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:22.055185 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a074479-9425-4d41-9ce6-dabd9b079d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:22.055450 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a074479-9425-4d41-9ce6-dabd9b079d18 admin admin] PATH_INFO: `/v3/users` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:22.056073 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a074479-9425-4d41-9ce6-dabd9b079d18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:22.058510 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a074479-9425-4d41-9ce6-dabd9b079d18 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:22.080605 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-858ec981-89a6-403c-8aef-129a2722cc0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9fadc8cbdc53422db8e82919ac88d4b5', 'request_id': 'req-858ec981-89a6-403c-8aef-129a2722cc0c'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:22.093689 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-858ec981-89a6-403c-8aef-129a2722cc0c admin admin] The token cache is being invalidate because project 9fadc8cbdc53422db8e82919ac88d4b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:10:22.093982 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-858ec981-89a6-403c-8aef-129a2722cc0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:22.094637 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-858ec981-89a6-403c-8aef-129a2722cc0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:22.096405 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1388/2774] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 09:10:21 2022] DELETE /identity/v3/projects/9fadc8cbdc53422db8e82919ac88d4b5 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:10:22.100943 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc47a53-d1c3-4086-b455-d48c358d00e0 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:22.101229 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc47a53-d1c3-4086-b455-d48c358d00e0 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:22.101504 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc47a53-d1c3-4086-b455-d48c358d00e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:22.123236 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-5cc47a53-d1c3-4086-b455-d48c358d00e0 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:10:22.124950 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-5cc47a53-d1c3-4086-b455-d48c358d00e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:22.135687 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1389/2775] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:10:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:10:22.144987 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23298b7a-4f5e-4b92-ac69-4ab1f9ba1318 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:22.149170 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23298b7a-4f5e-4b92-ac69-4ab1f9ba1318 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:22.149170 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23298b7a-4f5e-4b92-ac69-4ab1f9ba1318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:22.165720 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-23298b7a-4f5e-4b92-ac69-4ab1f9ba1318 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:10:22.166934 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-23298b7a-4f5e-4b92-ac69-4ab1f9ba1318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:22.173143 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1387/2776] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 09:10:21 2022] POST /identity/v3/users => generated 298 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:10:22.175762 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1390/2777] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:10:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:10:22.179762 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23298b7a-4f5e-4b92-ac69-4ab1f9ba1318 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:22.179913 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a074479-9425-4d41-9ce6-dabd9b079d18 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:22.181978 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-3a074479-9425-4d41-9ce6-dabd9b079d18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:22.189721 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23298b7a-4f5e-4b92-ac69-4ab1f9ba1318 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:22.193993 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a074479-9425-4d41-9ce6-dabd9b079d18 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:22.194465 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dcb3b9-7543-4058-b35b-d15caca68a58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:22.195199 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dcb3b9-7543-4058-b35b-d15caca68a58 admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:22.195481 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dcb3b9-7543-4058-b35b-d15caca68a58 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:22.195759 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dcb3b9-7543-4058-b35b-d15caca68a58 admin admin] PATH_INFO: `/v3/domains` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:22.196397 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dcb3b9-7543-4058-b35b-d15caca68a58 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:10:22.196765 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dcb3b9-7543-4058-b35b-d15caca68a58 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:22.198885 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dcb3b9-7543-4058-b35b-d15caca68a58 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:22.199535 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e877296-9fd3-4da5-b0b0-95960ad691b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:22.200426 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e877296-9fd3-4da5-b0b0-95960ad691b8 admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:22.200521 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e877296-9fd3-4da5-b0b0-95960ad691b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:22.200798 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e877296-9fd3-4da5-b0b0-95960ad691b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:22.201550 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e877296-9fd3-4da5-b0b0-95960ad691b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:22.203213 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e877296-9fd3-4da5-b0b0-95960ad691b8 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:22.212037 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1391/2778] 10.222.0.55 () {58 vars in 1153 bytes} [Wed Aug 24 09:10:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:10:22.214542 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1388/2779] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:10:22 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 09:10:22.220983 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dcb3b9-7543-4058-b35b-d15caca68a58 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:22.221139 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e877296-9fd3-4da5-b0b0-95960ad691b8 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:22.231039 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e877296-9fd3-4da5-b0b0-95960ad691b8 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:22.231209 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dcb3b9-7543-4058-b35b-d15caca68a58 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:22.235786 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48e6876-39cb-40c0-97b4-dc7d4cf54a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:22.235933 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e75ee0a-241c-44c8-84bf-0b5199f3629b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:22.236423 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e75ee0a-241c-44c8-84bf-0b5199f3629b admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:22.236578 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48e6876-39cb-40c0-97b4-dc7d4cf54a1a admin admin] REQUEST_METHOD: `PUT` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:22.236727 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48e6876-39cb-40c0-97b4-dc7d4cf54a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:22.236842 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e75ee0a-241c-44c8-84bf-0b5199f3629b admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:22.236997 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e75ee0a-241c-44c8-84bf-0b5199f3629b admin admin] PATH_INFO: `/v3/projects` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:22.237114 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48e6876-39cb-40c0-97b4-dc7d4cf54a1a admin admin] PATH_INFO: `/v3/projects/96de556c570541f7890d3fc2411e5600/users/e6869a639a7a47b796b9347db35608d6/roles/b870cd1acaa744e1ae1d58b9ac283821` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:22.237667 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e75ee0a-241c-44c8-84bf-0b5199f3629b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:22.239614 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e75ee0a-241c-44c8-84bf-0b5199f3629b admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:22.252650 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48e6876-39cb-40c0-97b4-dc7d4cf54a1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=96de556c570541f7890d3fc2411e5600, user_id=e6869a639a7a47b796b9347db35608d6, role_id=b870cd1acaa744e1ae1d58b9ac283821)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:22.255430 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48e6876-39cb-40c0-97b4-dc7d4cf54a1a admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:22.354551 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1392/2780] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 09:10:22 2022] POST /identity/v3/projects => generated 369 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:10:22.357213 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1389/2781] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 09:10:22 2022] PUT /identity/v3/projects/96de556c570541f7890d3fc2411e5600/users/e6869a639a7a47b796b9347db35608d6/roles/b870cd1acaa744e1ae1d58b9ac283821 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:10:22.358194 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e75ee0a-241c-44c8-84bf-0b5199f3629b admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:22.361367 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48e6876-39cb-40c0-97b4-dc7d4cf54a1a admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:22.438323 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48e6876-39cb-40c0-97b4-dc7d4cf54a1a admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:22.446713 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16d5137-173a-439e-98e6-3b5486f20106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:22.452792 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16d5137-173a-439e-98e6-3b5486f20106 admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:22.452792 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16d5137-173a-439e-98e6-3b5486f20106 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:22.452792 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16d5137-173a-439e-98e6-3b5486f20106 admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:22.452792 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16d5137-173a-439e-98e6-3b5486f20106 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:22.452792 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16d5137-173a-439e-98e6-3b5486f20106 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:22.453304 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e75ee0a-241c-44c8-84bf-0b5199f3629b admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:22.461381 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43002fa6-d009-48fc-96dc-16c939a1318d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:22.462081 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43002fa6-d009-48fc-96dc-16c939a1318d admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:22.462321 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43002fa6-d009-48fc-96dc-16c939a1318d admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:22.462437 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1390/2782] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:10:22 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:10:22.462591 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43002fa6-d009-48fc-96dc-16c939a1318d admin admin] PATH_INFO: `/v3/users` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:22.464779 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43002fa6-d009-48fc-96dc-16c939a1318d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:22.466591 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16d5137-173a-439e-98e6-3b5486f20106 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:22.471712 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43002fa6-d009-48fc-96dc-16c939a1318d admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:22.477606 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16d5137-173a-439e-98e6-3b5486f20106 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:22.483356 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d85f66-493c-4502-bef5-1110e0880603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:22.484401 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d85f66-493c-4502-bef5-1110e0880603 admin admin] REQUEST_METHOD: `PUT` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:22.484733 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d85f66-493c-4502-bef5-1110e0880603 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:22.485096 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d85f66-493c-4502-bef5-1110e0880603 admin admin] PATH_INFO: `/v3/projects/96de556c570541f7890d3fc2411e5600/users/e6869a639a7a47b796b9347db35608d6/roles/b870cd1acaa744e1ae1d58b9ac283821` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:22.487456 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d85f66-493c-4502-bef5-1110e0880603 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96de556c570541f7890d3fc2411e5600, user_id=e6869a639a7a47b796b9347db35608d6, role_id=b870cd1acaa744e1ae1d58b9ac283821)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:22.490215 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d85f66-493c-4502-bef5-1110e0880603 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:22.521072 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1391/2783] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 09:10:22 2022] PUT /identity/v3/projects/96de556c570541f7890d3fc2411e5600/users/e6869a639a7a47b796b9347db35608d6/roles/b870cd1acaa744e1ae1d58b9ac283821 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:10:22.523251 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7df18c-7a06-4f2d-9859-ed4c72940642 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:22.523629 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7df18c-7a06-4f2d-9859-ed4c72940642 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:22.523970 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7df18c-7a06-4f2d-9859-ed4c72940642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:22.557978 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:10:22.561187 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-bf7df18c-7a06-4f2d-9859-ed4c72940642 None None] MFA Rules not processed for user `e6869a639a7a47b796b9347db35608d6`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:10:22.565418 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1393/2784] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 09:10:22 2022] POST /identity/v3/users => generated 285 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:10:22.566070 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-bf7df18c-7a06-4f2d-9859-ed4c72940642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:22.566267 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43002fa6-d009-48fc-96dc-16c939a1318d admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:22.576591 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43002fa6-d009-48fc-96dc-16c939a1318d admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:22.585403 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:10:22.641716 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1392/2785] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:10:22 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:10:22.653227 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad24ad0-267e-4a0c-963e-c5cabeaab200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:22.654054 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad24ad0-267e-4a0c-963e-c5cabeaab200 admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:22.654425 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad24ad0-267e-4a0c-963e-c5cabeaab200 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:22.654766 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad24ad0-267e-4a0c-963e-c5cabeaab200 admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:22.654980 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7df18c-7a06-4f2d-9859-ed4c72940642 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:22.656213 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad24ad0-267e-4a0c-963e-c5cabeaab200 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:22.657361 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-bf7df18c-7a06-4f2d-9859-ed4c72940642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:22.657998 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad24ad0-267e-4a0c-963e-c5cabeaab200 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:22.670140 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1394/2786] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:10:22 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:10:22.676513 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad24ad0-267e-4a0c-963e-c5cabeaab200 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:22.687403 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad24ad0-267e-4a0c-963e-c5cabeaab200 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:22.693375 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e012e30-5f4c-4aba-9e62-b74e40f4aeb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:22.694403 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e012e30-5f4c-4aba-9e62-b74e40f4aeb4 admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:22.694693 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e012e30-5f4c-4aba-9e62-b74e40f4aeb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:22.694970 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e012e30-5f4c-4aba-9e62-b74e40f4aeb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:22.695615 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e012e30-5f4c-4aba-9e62-b74e40f4aeb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:22.697363 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e012e30-5f4c-4aba-9e62-b74e40f4aeb4 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:22.708426 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1395/2787] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:10:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 09:10:22.712628 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e012e30-5f4c-4aba-9e62-b74e40f4aeb4 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:22.723616 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e012e30-5f4c-4aba-9e62-b74e40f4aeb4 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:22.730419 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09909a2-dc63-4b4f-950f-68aa32c7d0e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:22.730735 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7df18c-7a06-4f2d-9859-ed4c72940642 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:22.731210 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09909a2-dc63-4b4f-950f-68aa32c7d0e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:22.731348 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09909a2-dc63-4b4f-950f-68aa32c7d0e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:22.731626 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09909a2-dc63-4b4f-950f-68aa32c7d0e7 admin admin] PATH_INFO: `/v3/projects/f3af1a6c856340efa227b5925f6c2d7b/users/0b9fba36ffcd45a0881a54a82eb704a8/roles/b674fd0a592e4e0cada811bba5ceea4d` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:22.735944 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d1e5c6-4dd3-4de1-9ab9-9395450d4278 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbb9ac667f04c35a63b2689df18efed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:22.736718 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d1e5c6-4dd3-4de1-9ab9-9395450d4278 service cinder] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:22.737193 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d1e5c6-4dd3-4de1-9ab9-9395450d4278 service cinder] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:22.737460 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d1e5c6-4dd3-4de1-9ab9-9395450d4278 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:22.748131 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09909a2-dc63-4b4f-950f-68aa32c7d0e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3af1a6c856340efa227b5925f6c2d7b, user_id=0b9fba36ffcd45a0881a54a82eb704a8, role_id=b674fd0a592e4e0cada811bba5ceea4d)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:22.748875 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d1e5c6-4dd3-4de1-9ab9-9395450d4278 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:22.750309 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d1e5c6-4dd3-4de1-9ab9-9395450d4278 service cinder] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:22.751373 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09909a2-dc63-4b4f-950f-68aa32c7d0e7 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:22.760375 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1393/2788] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:10:22 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 09:10:22.836966 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1396/2789] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 09:10:22 2022] PUT /identity/v3/projects/f3af1a6c856340efa227b5925f6c2d7b/users/0b9fba36ffcd45a0881a54a82eb704a8/roles/b674fd0a592e4e0cada811bba5ceea4d => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:10:22.857432 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e7ea35-7379-4749-92dc-390384ea4702 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:22.857432 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e7ea35-7379-4749-92dc-390384ea4702 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:22.857432 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e7ea35-7379-4749-92dc-390384ea4702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:22.864701 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09909a2-dc63-4b4f-950f-68aa32c7d0e7 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:22.881843 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-39e7ea35-7379-4749-92dc-390384ea4702 None None] MFA Rules not processed for user `0b9fba36ffcd45a0881a54a82eb704a8`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:10:22.883097 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-39e7ea35-7379-4749-92dc-390384ea4702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:22.945598 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09909a2-dc63-4b4f-950f-68aa32c7d0e7 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:22.951080 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dd5848-b7e4-45de-a8dc-b9e9e1b32ad6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbb9ac667f04c35a63b2689df18efed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:22.951887 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dd5848-b7e4-45de-a8dc-b9e9e1b32ad6 service cinder] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:22.952380 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dd5848-b7e4-45de-a8dc-b9e9e1b32ad6 service cinder] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:22.952847 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dd5848-b7e4-45de-a8dc-b9e9e1b32ad6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:22.956319 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-37dd5848-b7e4-45de-a8dc-b9e9e1b32ad6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:22.959412 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1394/2790] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:10:22 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:10:22.973733 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e7ea35-7379-4749-92dc-390384ea4702 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:22.975129 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-39e7ea35-7379-4749-92dc-390384ea4702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:23.015400 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:10:23.045657 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dd5848-b7e4-45de-a8dc-b9e9e1b32ad6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:23.047512 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dd5848-b7e4-45de-a8dc-b9e9e1b32ad6 service cinder] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:23.058992 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1397/2791] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:10:22 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:10:23.078344 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e7ea35-7379-4749-92dc-390384ea4702 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:23.082163 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21860200-d930-4288-ab19-fa343af4582a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5512043ca9a7421b821741ca88d50b07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:23.082843 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21860200-d930-4288-ab19-fa343af4582a service glance] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:23.084957 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21860200-d930-4288-ab19-fa343af4582a service glance] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:23.084957 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21860200-d930-4288-ab19-fa343af4582a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:23.095156 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21860200-d930-4288-ab19-fa343af4582a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:23.096660 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21860200-d930-4288-ab19-fa343af4582a service glance] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:23.107025 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1395/2792] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:10:22 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 09:10:25.424800 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dd5848-b7e4-45de-a8dc-b9e9e1b32ad6 service cinder] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:25.426323 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-37dd5848-b7e4-45de-a8dc-b9e9e1b32ad6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:25.460597 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:10:25.497270 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dd5848-b7e4-45de-a8dc-b9e9e1b32ad6 service cinder] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:25.502326 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b2c1a6-b694-4aa3-8352-7022cb79c655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:25.502969 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b2c1a6-b694-4aa3-8352-7022cb79c655 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:25.503250 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b2c1a6-b694-4aa3-8352-7022cb79c655 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:25.503530 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b2c1a6-b694-4aa3-8352-7022cb79c655 admin admin] PATH_INFO: `/v3/users/c2c3752027f342a489e6eb9d3b540819` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:25.504715 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b2c1a6-b694-4aa3-8352-7022cb79c655 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2c3752027f342a489e6eb9d3b540819)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:25.506710 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b2c1a6-b694-4aa3-8352-7022cb79c655 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:25.683924 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-c5b2c1a6-b694-4aa3-8352-7022cb79c655 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2c3752027f342a489e6eb9d3b540819'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:25.684598 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-c5b2c1a6-b694-4aa3-8352-7022cb79c655 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2c3752027f342a489e6eb9d3b540819'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:25.685209 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-c5b2c1a6-b694-4aa3-8352-7022cb79c655 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2c3752027f342a489e6eb9d3b540819'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:25.699346 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-c5b2c1a6-b694-4aa3-8352-7022cb79c655 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2c3752027f342a489e6eb9d3b540819'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:25.754525 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-c5b2c1a6-b694-4aa3-8352-7022cb79c655 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2c3752027f342a489e6eb9d3b540819'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:25.769500 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1398/2793] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 09:10:25 2022] DELETE /identity/v3/users/c2c3752027f342a489e6eb9d3b540819 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:10:26.013832 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21860200-d930-4288-ab19-fa343af4582a service glance] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:26.028663 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21860200-d930-4288-ab19-fa343af4582a service glance] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:26.088964 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e854c177-ba71-4cca-b00b-a717819d4830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:26.089629 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e854c177-ba71-4cca-b00b-a717819d4830 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:26.089890 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e854c177-ba71-4cca-b00b-a717819d4830 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:26.090164 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e854c177-ba71-4cca-b00b-a717819d4830 admin admin] PATH_INFO: `/v3/projects/1f01e21018a54220ba1e54c6b7bfc400` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:26.091305 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e854c177-ba71-4cca-b00b-a717819d4830 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f01e21018a54220ba1e54c6b7bfc400)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:26.093243 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e854c177-ba71-4cca-b00b-a717819d4830 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:26.195230 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b2c1a6-b694-4aa3-8352-7022cb79c655 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:26.196707 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-c5b2c1a6-b694-4aa3-8352-7022cb79c655 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:26.205768 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-e854c177-ba71-4cca-b00b-a717819d4830 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f01e21018a54220ba1e54c6b7bfc400', 'request_id': 'req-e854c177-ba71-4cca-b00b-a717819d4830'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:26.214002 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b2c1a6-b694-4aa3-8352-7022cb79c655 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:26.282063 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beabd9f0-f0d6-407a-945c-9195d97050e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:26.282867 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beabd9f0-f0d6-407a-945c-9195d97050e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:26.283154 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beabd9f0-f0d6-407a-945c-9195d97050e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:26.283429 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beabd9f0-f0d6-407a-945c-9195d97050e1 admin admin] PATH_INFO: `/v3/users/0fb57b7fb7f34202ae37af788309736a` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:26.284617 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beabd9f0-f0d6-407a-945c-9195d97050e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fb57b7fb7f34202ae37af788309736a)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:26.286643 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beabd9f0-f0d6-407a-945c-9195d97050e1 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:26.315274 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-e854c177-ba71-4cca-b00b-a717819d4830 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f01e21018a54220ba1e54c6b7bfc400', 'request_id': 'req-e854c177-ba71-4cca-b00b-a717819d4830'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:26.323019 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-e854c177-ba71-4cca-b00b-a717819d4830 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f01e21018a54220ba1e54c6b7bfc400', 'request_id': 'req-e854c177-ba71-4cca-b00b-a717819d4830'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:26.334530 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-e854c177-ba71-4cca-b00b-a717819d4830 admin admin] The token cache is being invalidate because project 1f01e21018a54220ba1e54c6b7bfc400 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:10:26.334913 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-e854c177-ba71-4cca-b00b-a717819d4830 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:26.336373 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-e854c177-ba71-4cca-b00b-a717819d4830 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:26.338071 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1396/2794] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 09:10:26 2022] DELETE /identity/v3/projects/1f01e21018a54220ba1e54c6b7bfc400 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:10:26.342343 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e854c177-ba71-4cca-b00b-a717819d4830 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:26.343896 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-e854c177-ba71-4cca-b00b-a717819d4830 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:26.357458 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e854c177-ba71-4cca-b00b-a717819d4830 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:26.362321 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c06f3a0-66a0-470e-89d5-72993c58b2fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:26.362948 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c06f3a0-66a0-470e-89d5-72993c58b2fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:26.363225 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c06f3a0-66a0-470e-89d5-72993c58b2fd admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:26.363497 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c06f3a0-66a0-470e-89d5-72993c58b2fd admin admin] PATH_INFO: `/v3/users/fc533493bf4e4f9cb3f5079d7b3ab3f2` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:26.379531 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c06f3a0-66a0-470e-89d5-72993c58b2fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc533493bf4e4f9cb3f5079d7b3ab3f2)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:26.381499 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c06f3a0-66a0-470e-89d5-72993c58b2fd admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:26.466089 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:10:26.467485 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-beabd9f0-f0d6-407a-945c-9195d97050e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fb57b7fb7f34202ae37af788309736a'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:26.468147 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-beabd9f0-f0d6-407a-945c-9195d97050e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fb57b7fb7f34202ae37af788309736a'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:26.468785 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-beabd9f0-f0d6-407a-945c-9195d97050e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fb57b7fb7f34202ae37af788309736a'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:26.470630 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:10:26.483360 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-beabd9f0-f0d6-407a-945c-9195d97050e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fb57b7fb7f34202ae37af788309736a'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:26.557354 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-3c06f3a0-66a0-470e-89d5-72993c58b2fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc533493bf4e4f9cb3f5079d7b3ab3f2'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:26.558045 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-3c06f3a0-66a0-470e-89d5-72993c58b2fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc533493bf4e4f9cb3f5079d7b3ab3f2'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:26.558668 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-3c06f3a0-66a0-470e-89d5-72993c58b2fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc533493bf4e4f9cb3f5079d7b3ab3f2'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:26.566292 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-beabd9f0-f0d6-407a-945c-9195d97050e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fb57b7fb7f34202ae37af788309736a'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:26.571650 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-3c06f3a0-66a0-470e-89d5-72993c58b2fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc533493bf4e4f9cb3f5079d7b3ab3f2'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:26.579550 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1399/2795] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 09:10:26 2022] DELETE /identity/v3/users/0fb57b7fb7f34202ae37af788309736a => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:10:26.584089 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e808dd16-8561-462a-bcf7-05ccae8c5328 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:26.584375 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e808dd16-8561-462a-bcf7-05ccae8c5328 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:26.584857 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e808dd16-8561-462a-bcf7-05ccae8c5328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:26.605052 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-e808dd16-8561-462a-bcf7-05ccae8c5328 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:10:26.606277 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-e808dd16-8561-462a-bcf7-05ccae8c5328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:26.640396 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-3c06f3a0-66a0-470e-89d5-72993c58b2fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc533493bf4e4f9cb3f5079d7b3ab3f2'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:26.655134 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1397/2796] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 09:10:26 2022] DELETE /identity/v3/users/fc533493bf4e4f9cb3f5079d7b3ab3f2 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:10:26.673901 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1400/2797] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:10:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:10:26.693590 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c06f3a0-66a0-470e-89d5-72993c58b2fd admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:26.695314 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-3c06f3a0-66a0-470e-89d5-72993c58b2fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:26.767212 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c06f3a0-66a0-470e-89d5-72993c58b2fd admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:26.770835 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d08fc24-1f65-402f-849b-1efa29894c40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:26.771434 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d08fc24-1f65-402f-849b-1efa29894c40 service neutron] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:26.771708 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d08fc24-1f65-402f-849b-1efa29894c40 service neutron] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:26.771974 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d08fc24-1f65-402f-849b-1efa29894c40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:26.836421 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d08fc24-1f65-402f-849b-1efa29894c40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:26.838491 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d08fc24-1f65-402f-849b-1efa29894c40 service neutron] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:26.848608 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1398/2798] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:10:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 09:10:26.918121 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e808dd16-8561-462a-bcf7-05ccae8c5328 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:26.919683 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-e808dd16-8561-462a-bcf7-05ccae8c5328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:26.935557 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e808dd16-8561-462a-bcf7-05ccae8c5328 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:26.945580 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a47d9b-6b51-466f-8175-4cf89e5f23bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:26.945810 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a47d9b-6b51-466f-8175-4cf89e5f23bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:26.946001 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a47d9b-6b51-466f-8175-4cf89e5f23bf admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:26.946279 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a47d9b-6b51-466f-8175-4cf89e5f23bf admin admin] PATH_INFO: `/v3/projects/19adc18f8f2c4de8b5d90c47645aa6f7` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:26.960655 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a47d9b-6b51-466f-8175-4cf89e5f23bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=19adc18f8f2c4de8b5d90c47645aa6f7)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:26.965311 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a47d9b-6b51-466f-8175-4cf89e5f23bf admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:27.098155 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-57a47d9b-6b51-466f-8175-4cf89e5f23bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19adc18f8f2c4de8b5d90c47645aa6f7', 'request_id': 'req-57a47d9b-6b51-466f-8175-4cf89e5f23bf'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:27.104357 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-57a47d9b-6b51-466f-8175-4cf89e5f23bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19adc18f8f2c4de8b5d90c47645aa6f7', 'request_id': 'req-57a47d9b-6b51-466f-8175-4cf89e5f23bf'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:27.183763 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-57a47d9b-6b51-466f-8175-4cf89e5f23bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19adc18f8f2c4de8b5d90c47645aa6f7', 'request_id': 'req-57a47d9b-6b51-466f-8175-4cf89e5f23bf'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:27.201604 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-57a47d9b-6b51-466f-8175-4cf89e5f23bf admin admin] The token cache is being invalidate because project 19adc18f8f2c4de8b5d90c47645aa6f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:10:27.202026 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-57a47d9b-6b51-466f-8175-4cf89e5f23bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:27.202792 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-57a47d9b-6b51-466f-8175-4cf89e5f23bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:27.216869 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1401/2799] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 09:10:26 2022] DELETE /identity/v3/projects/19adc18f8f2c4de8b5d90c47645aa6f7 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:10:27.291047 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d08fc24-1f65-402f-849b-1efa29894c40 service neutron] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:27.292701 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-1d08fc24-1f65-402f-849b-1efa29894c40 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:27.320166 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d08fc24-1f65-402f-849b-1efa29894c40 service neutron] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:27.383347 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:10:27.391559 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc995c73-e0cb-4d20-b42c-426e97eefdb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:27.392209 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc995c73-e0cb-4d20-b42c-426e97eefdb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:27.392485 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc995c73-e0cb-4d20-b42c-426e97eefdb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:27.392784 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc995c73-e0cb-4d20-b42c-426e97eefdb9 admin admin] PATH_INFO: `/v3/projects/bf658344bfc44fed9bb250573ed1cb28` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:27.393929 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc995c73-e0cb-4d20-b42c-426e97eefdb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf658344bfc44fed9bb250573ed1cb28)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:27.398040 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc995c73-e0cb-4d20-b42c-426e97eefdb9 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:27.509334 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-bc995c73-e0cb-4d20-b42c-426e97eefdb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf658344bfc44fed9bb250573ed1cb28', 'request_id': 'req-bc995c73-e0cb-4d20-b42c-426e97eefdb9'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:27.577757 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-bc995c73-e0cb-4d20-b42c-426e97eefdb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf658344bfc44fed9bb250573ed1cb28', 'request_id': 'req-bc995c73-e0cb-4d20-b42c-426e97eefdb9'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:27.585361 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-bc995c73-e0cb-4d20-b42c-426e97eefdb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf658344bfc44fed9bb250573ed1cb28', 'request_id': 'req-bc995c73-e0cb-4d20-b42c-426e97eefdb9'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:27.597313 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-bc995c73-e0cb-4d20-b42c-426e97eefdb9 admin admin] The token cache is being invalidate because project bf658344bfc44fed9bb250573ed1cb28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:10:27.603742 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-bc995c73-e0cb-4d20-b42c-426e97eefdb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:27.603742 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-bc995c73-e0cb-4d20-b42c-426e97eefdb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:27.603742 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1399/2800] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 09:10:27 2022] DELETE /identity/v3/projects/bf658344bfc44fed9bb250573ed1cb28 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:10:27.608966 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a47d9b-6b51-466f-8175-4cf89e5f23bf admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:27.610281 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-57a47d9b-6b51-466f-8175-4cf89e5f23bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:27.625060 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a47d9b-6b51-466f-8175-4cf89e5f23bf admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:27.665415 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:10:27.685589 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18afd89-55d1-4952-b35c-d06df6c56be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:27.686225 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18afd89-55d1-4952-b35c-d06df6c56be4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:27.686502 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18afd89-55d1-4952-b35c-d06df6c56be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:27.686775 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18afd89-55d1-4952-b35c-d06df6c56be4 admin admin] PATH_INFO: `/v3/users/e6869a639a7a47b796b9347db35608d6` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:27.687930 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18afd89-55d1-4952-b35c-d06df6c56be4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6869a639a7a47b796b9347db35608d6)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:27.689869 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18afd89-55d1-4952-b35c-d06df6c56be4 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:27.871177 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-b18afd89-55d1-4952-b35c-d06df6c56be4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6869a639a7a47b796b9347db35608d6'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:27.871869 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-b18afd89-55d1-4952-b35c-d06df6c56be4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6869a639a7a47b796b9347db35608d6'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:27.872588 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-b18afd89-55d1-4952-b35c-d06df6c56be4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6869a639a7a47b796b9347db35608d6'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:27.886107 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-b18afd89-55d1-4952-b35c-d06df6c56be4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6869a639a7a47b796b9347db35608d6'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:27.971400 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-b18afd89-55d1-4952-b35c-d06df6c56be4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6869a639a7a47b796b9347db35608d6'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:27.985136 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1402/2801] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 09:10:27 2022] DELETE /identity/v3/users/e6869a639a7a47b796b9347db35608d6 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:10:28.499813 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc995c73-e0cb-4d20-b42c-426e97eefdb9 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:28.501440 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-bc995c73-e0cb-4d20-b42c-426e97eefdb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:28.520315 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc995c73-e0cb-4d20-b42c-426e97eefdb9 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:28.577345 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcf102f-2e5c-4e5c-9add-09d5f0dbe536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:28.578039 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcf102f-2e5c-4e5c-9add-09d5f0dbe536 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:28.578255 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcf102f-2e5c-4e5c-9add-09d5f0dbe536 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:28.578540 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcf102f-2e5c-4e5c-9add-09d5f0dbe536 admin admin] PATH_INFO: `/v3/projects/96de556c570541f7890d3fc2411e5600` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:28.579647 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcf102f-2e5c-4e5c-9add-09d5f0dbe536 admin admin] RBAC: Authorizing `identity:delete_project(project_id=96de556c570541f7890d3fc2411e5600)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:28.581592 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcf102f-2e5c-4e5c-9add-09d5f0dbe536 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:28.715639 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-bfcf102f-2e5c-4e5c-9add-09d5f0dbe536 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96de556c570541f7890d3fc2411e5600', 'request_id': 'req-bfcf102f-2e5c-4e5c-9add-09d5f0dbe536'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:28.783346 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-bfcf102f-2e5c-4e5c-9add-09d5f0dbe536 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96de556c570541f7890d3fc2411e5600', 'request_id': 'req-bfcf102f-2e5c-4e5c-9add-09d5f0dbe536'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:28.790386 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-bfcf102f-2e5c-4e5c-9add-09d5f0dbe536 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96de556c570541f7890d3fc2411e5600', 'request_id': 'req-bfcf102f-2e5c-4e5c-9add-09d5f0dbe536'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:28.801491 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-bfcf102f-2e5c-4e5c-9add-09d5f0dbe536 admin admin] The token cache is being invalidate because project 96de556c570541f7890d3fc2411e5600 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:10:28.801792 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-bfcf102f-2e5c-4e5c-9add-09d5f0dbe536 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:28.802478 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-bfcf102f-2e5c-4e5c-9add-09d5f0dbe536 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:28.803988 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1400/2802] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 09:10:28 2022] DELETE /identity/v3/projects/96de556c570541f7890d3fc2411e5600 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:10:28.809499 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5af69c-bb36-4f87-ba87-ed10006b37d4 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:28.809970 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5af69c-bb36-4f87-ba87-ed10006b37d4 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:28.810096 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5af69c-bb36-4f87-ba87-ed10006b37d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:28.831598 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-5c5af69c-bb36-4f87-ba87-ed10006b37d4 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:10:28.832926 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-5c5af69c-bb36-4f87-ba87-ed10006b37d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:28.894032 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1403/2803] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:10:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:10:28.901828 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45edb5a-4014-404f-a6ae-f7871112afb8 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:28.902109 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45edb5a-4014-404f-a6ae-f7871112afb8 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:28.902406 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45edb5a-4014-404f-a6ae-f7871112afb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:28.923304 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-b45edb5a-4014-404f-a6ae-f7871112afb8 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:10:28.924553 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-b45edb5a-4014-404f-a6ae-f7871112afb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:28.933446 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1401/2804] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:10:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:10:28.938081 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5af69c-bb36-4f87-ba87-ed10006b37d4 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:28.949203 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5af69c-bb36-4f87-ba87-ed10006b37d4 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:28.953919 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff857ef4-0ad9-4edc-a9ff-26c2fcd6a174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:28.954786 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff857ef4-0ad9-4edc-a9ff-26c2fcd6a174 admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:28.955029 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff857ef4-0ad9-4edc-a9ff-26c2fcd6a174 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:28.955324 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff857ef4-0ad9-4edc-a9ff-26c2fcd6a174 admin admin] PATH_INFO: `/v3/domains` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:28.956053 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff857ef4-0ad9-4edc-a9ff-26c2fcd6a174 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:10:28.956357 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff857ef4-0ad9-4edc-a9ff-26c2fcd6a174 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:28.958081 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff857ef4-0ad9-4edc-a9ff-26c2fcd6a174 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:28.970116 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1404/2805] 10.222.0.55 () {58 vars in 1153 bytes} [Wed Aug 24 09:10:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:10:28.974441 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45edb5a-4014-404f-a6ae-f7871112afb8 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:28.984041 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45edb5a-4014-404f-a6ae-f7871112afb8 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:28.988745 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0bc46c-9ec6-44af-b803-8e3b336c7a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:28.989332 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0bc46c-9ec6-44af-b803-8e3b336c7a49 admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:28.989608 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0bc46c-9ec6-44af-b803-8e3b336c7a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:28.989960 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0bc46c-9ec6-44af-b803-8e3b336c7a49 admin admin] PATH_INFO: `/v3/projects` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:28.990490 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0bc46c-9ec6-44af-b803-8e3b336c7a49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:28.992446 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0bc46c-9ec6-44af-b803-8e3b336c7a49 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:28.998230 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:10:29.093824 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1402/2806] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 09:10:28 2022] POST /identity/v3/projects => generated 385 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:10:29.098105 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff857ef4-0ad9-4edc-a9ff-26c2fcd6a174 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:29.109182 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff857ef4-0ad9-4edc-a9ff-26c2fcd6a174 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:29.155056 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:10:29.169866 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813f0299-9226-4ccc-b0cf-aa3346b82494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:29.170477 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813f0299-9226-4ccc-b0cf-aa3346b82494 admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:29.170756 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813f0299-9226-4ccc-b0cf-aa3346b82494 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:29.171028 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813f0299-9226-4ccc-b0cf-aa3346b82494 admin admin] PATH_INFO: `/v3/users` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:29.171676 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813f0299-9226-4ccc-b0cf-aa3346b82494 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:29.173631 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813f0299-9226-4ccc-b0cf-aa3346b82494 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:29.269560 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1405/2807] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 09:10:29 2022] POST /identity/v3/users => generated 293 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:10:29.274003 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0bc46c-9ec6-44af-b803-8e3b336c7a49 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:29.284430 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0bc46c-9ec6-44af-b803-8e3b336c7a49 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:29.289391 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce94e8e-d5b8-4e8d-acc3-1a59f22cb40d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:29.290039 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce94e8e-d5b8-4e8d-acc3-1a59f22cb40d admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:29.290263 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce94e8e-d5b8-4e8d-acc3-1a59f22cb40d admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:29.290534 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce94e8e-d5b8-4e8d-acc3-1a59f22cb40d admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:29.291215 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce94e8e-d5b8-4e8d-acc3-1a59f22cb40d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:29.293342 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce94e8e-d5b8-4e8d-acc3-1a59f22cb40d admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:29.303900 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1403/2808] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:10:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:10:29.308596 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813f0299-9226-4ccc-b0cf-aa3346b82494 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:29.319959 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813f0299-9226-4ccc-b0cf-aa3346b82494 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:29.324638 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d77d60-8203-4a20-be68-9b41222a9f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:29.325255 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d77d60-8203-4a20-be68-9b41222a9f1f admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:29.325534 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d77d60-8203-4a20-be68-9b41222a9f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:29.325808 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d77d60-8203-4a20-be68-9b41222a9f1f admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:29.326451 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d77d60-8203-4a20-be68-9b41222a9f1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:29.328038 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d77d60-8203-4a20-be68-9b41222a9f1f admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:29.339757 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1406/2809] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:10:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:10:29.343637 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce94e8e-d5b8-4e8d-acc3-1a59f22cb40d admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:29.353609 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce94e8e-d5b8-4e8d-acc3-1a59f22cb40d admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:29.358967 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9564fb86-1310-47e7-ba46-19084480218d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:29.359625 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9564fb86-1310-47e7-ba46-19084480218d admin admin] REQUEST_METHOD: `PUT` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:29.359903 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9564fb86-1310-47e7-ba46-19084480218d admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:29.360193 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9564fb86-1310-47e7-ba46-19084480218d admin admin] PATH_INFO: `/v3/projects/b73fa10e2211420bbe0909ad482ab4e6/users/a125159ca6cb43169e9c336ae9f85f48/roles/b674fd0a592e4e0cada811bba5ceea4d` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:29.376594 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9564fb86-1310-47e7-ba46-19084480218d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b73fa10e2211420bbe0909ad482ab4e6, user_id=a125159ca6cb43169e9c336ae9f85f48, role_id=b674fd0a592e4e0cada811bba5ceea4d)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:29.379226 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9564fb86-1310-47e7-ba46-19084480218d admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:29.449960 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1404/2810] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 09:10:29 2022] PUT /identity/v3/projects/b73fa10e2211420bbe0909ad482ab4e6/users/a125159ca6cb43169e9c336ae9f85f48/roles/b674fd0a592e4e0cada811bba5ceea4d => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:10:29.457755 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4209507-54b2-4e49-9823-31d55779e35b None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:29.458041 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4209507-54b2-4e49-9823-31d55779e35b None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:29.458310 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4209507-54b2-4e49-9823-31d55779e35b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:29.488455 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-d4209507-54b2-4e49-9823-31d55779e35b None None] MFA Rules not processed for user `a125159ca6cb43169e9c336ae9f85f48`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:10:29.489828 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-d4209507-54b2-4e49-9823-31d55779e35b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:29.559929 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1407/2811] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:10:29 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:10:29.565111 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9564fb86-1310-47e7-ba46-19084480218d admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:29.627849 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9564fb86-1310-47e7-ba46-19084480218d admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:29.632611 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad50e3d6-c32e-40b0-a8b8-98e1a2e84764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:29.633210 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad50e3d6-c32e-40b0-a8b8-98e1a2e84764 admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:29.633488 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad50e3d6-c32e-40b0-a8b8-98e1a2e84764 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:29.633756 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad50e3d6-c32e-40b0-a8b8-98e1a2e84764 admin admin] PATH_INFO: `/v3/projects` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:29.634366 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad50e3d6-c32e-40b0-a8b8-98e1a2e84764 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:29.636239 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad50e3d6-c32e-40b0-a8b8-98e1a2e84764 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:29.734233 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1405/2812] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 09:10:29 2022] POST /identity/v3/projects => generated 385 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:10:29.738593 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4209507-54b2-4e49-9823-31d55779e35b None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:29.749289 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4209507-54b2-4e49-9823-31d55779e35b None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:29.931191 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c3d0c5-bc6e-40f3-8463-c30440c30d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:29.931901 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c3d0c5-bc6e-40f3-8463-c30440c30d69 admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:29.932032 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c3d0c5-bc6e-40f3-8463-c30440c30d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:29.932234 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c3d0c5-bc6e-40f3-8463-c30440c30d69 admin admin] PATH_INFO: `/v3/users` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:29.933602 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c3d0c5-bc6e-40f3-8463-c30440c30d69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:29.935809 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c3d0c5-bc6e-40f3-8463-c30440c30d69 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:30.035588 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1408/2813] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 09:10:29 2022] POST /identity/v3/users => generated 299 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:10:30.039740 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad50e3d6-c32e-40b0-a8b8-98e1a2e84764 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:30.050291 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad50e3d6-c32e-40b0-a8b8-98e1a2e84764 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:30.055146 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec03aa5b-90fe-46f2-a79a-1f8cda524a0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:30.055758 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec03aa5b-90fe-46f2-a79a-1f8cda524a0a admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:30.056036 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec03aa5b-90fe-46f2-a79a-1f8cda524a0a admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:30.056309 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec03aa5b-90fe-46f2-a79a-1f8cda524a0a admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:30.056988 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec03aa5b-90fe-46f2-a79a-1f8cda524a0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:30.058613 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec03aa5b-90fe-46f2-a79a-1f8cda524a0a admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:30.069336 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1406/2814] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:10:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:10:30.074816 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c3d0c5-bc6e-40f3-8463-c30440c30d69 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:30.085014 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c3d0c5-bc6e-40f3-8463-c30440c30d69 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:30.090310 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526cb4b4-7424-4cb9-8831-edb262559de9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:30.090959 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526cb4b4-7424-4cb9-8831-edb262559de9 admin admin] REQUEST_METHOD: `PUT` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:30.091234 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526cb4b4-7424-4cb9-8831-edb262559de9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:30.091504 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526cb4b4-7424-4cb9-8831-edb262559de9 admin admin] PATH_INFO: `/v3/projects/f656770d7e53463cadc8d95e0e6bb493/users/74a1290cd2a24063a21013b22238818b/roles/b870cd1acaa744e1ae1d58b9ac283821` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:30.106543 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526cb4b4-7424-4cb9-8831-edb262559de9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f656770d7e53463cadc8d95e0e6bb493, user_id=74a1290cd2a24063a21013b22238818b, role_id=b870cd1acaa744e1ae1d58b9ac283821)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:30.109247 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526cb4b4-7424-4cb9-8831-edb262559de9 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:30.177728 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1409/2815] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 09:10:30 2022] PUT /identity/v3/projects/f656770d7e53463cadc8d95e0e6bb493/users/74a1290cd2a24063a21013b22238818b/roles/b870cd1acaa744e1ae1d58b9ac283821 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:10:30.181778 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec03aa5b-90fe-46f2-a79a-1f8cda524a0a admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:30.192707 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec03aa5b-90fe-46f2-a79a-1f8cda524a0a admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:30.251703 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8394d5cc-b3da-4b8c-9977-63407f9c1bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:30.252413 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8394d5cc-b3da-4b8c-9977-63407f9c1bf0 admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:30.252757 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8394d5cc-b3da-4b8c-9977-63407f9c1bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:30.253026 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8394d5cc-b3da-4b8c-9977-63407f9c1bf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:30.253657 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8394d5cc-b3da-4b8c-9977-63407f9c1bf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:30.255457 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8394d5cc-b3da-4b8c-9977-63407f9c1bf0 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:30.267573 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1407/2816] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:10:30 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:10:30.271554 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526cb4b4-7424-4cb9-8831-edb262559de9 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:30.282059 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526cb4b4-7424-4cb9-8831-edb262559de9 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:30.286598 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e487d6a7-7702-42ca-b596-01418ca2aca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:30.287500 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e487d6a7-7702-42ca-b596-01418ca2aca5 admin admin] REQUEST_METHOD: `PUT` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:30.287782 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e487d6a7-7702-42ca-b596-01418ca2aca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:30.288048 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e487d6a7-7702-42ca-b596-01418ca2aca5 admin admin] PATH_INFO: `/v3/projects/f656770d7e53463cadc8d95e0e6bb493/users/74a1290cd2a24063a21013b22238818b/roles/b870cd1acaa744e1ae1d58b9ac283821` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:30.289985 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e487d6a7-7702-42ca-b596-01418ca2aca5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f656770d7e53463cadc8d95e0e6bb493, user_id=74a1290cd2a24063a21013b22238818b, role_id=b870cd1acaa744e1ae1d58b9ac283821)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:30.292738 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e487d6a7-7702-42ca-b596-01418ca2aca5 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:30.302918 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1410/2817] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 09:10:30 2022] PUT /identity/v3/projects/f656770d7e53463cadc8d95e0e6bb493/users/74a1290cd2a24063a21013b22238818b/roles/b870cd1acaa744e1ae1d58b9ac283821 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:10:30.314748 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cb61cc-e4b4-42b0-a96f-a21e804b5cdb None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:30.315040 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cb61cc-e4b4-42b0-a96f-a21e804b5cdb None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:30.315362 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cb61cc-e4b4-42b0-a96f-a21e804b5cdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:30.346930 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-a3cb61cc-e4b4-42b0-a96f-a21e804b5cdb None None] MFA Rules not processed for user `74a1290cd2a24063a21013b22238818b`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:10:30.348199 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-a3cb61cc-e4b4-42b0-a96f-a21e804b5cdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:30.360728 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:10:30.426502 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1408/2818] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:10:30 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:10:30.438122 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e487d6a7-7702-42ca-b596-01418ca2aca5 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:30.439925 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-e487d6a7-7702-42ca-b596-01418ca2aca5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:30.451251 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:10:30.500199 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e487d6a7-7702-42ca-b596-01418ca2aca5 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:30.504395 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d090fd39-82f9-4d4e-bf6f-b2626820c7d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbb9ac667f04c35a63b2689df18efed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:30.505095 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d090fd39-82f9-4d4e-bf6f-b2626820c7d4 service cinder] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:30.505374 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d090fd39-82f9-4d4e-bf6f-b2626820c7d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:30.505644 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d090fd39-82f9-4d4e-bf6f-b2626820c7d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:30.515808 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d090fd39-82f9-4d4e-bf6f-b2626820c7d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:30.517272 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d090fd39-82f9-4d4e-bf6f-b2626820c7d4 service cinder] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:30.527142 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1411/2819] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:10:30 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:10:31.604682 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cb61cc-e4b4-42b0-a96f-a21e804b5cdb None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:31.669368 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cb61cc-e4b4-42b0-a96f-a21e804b5cdb None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:31.673970 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac2e29a-ef7a-48c9-94e0-d8d6cca4e197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:31.674593 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac2e29a-ef7a-48c9-94e0-d8d6cca4e197 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:31.674872 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac2e29a-ef7a-48c9-94e0-d8d6cca4e197 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:31.675145 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac2e29a-ef7a-48c9-94e0-d8d6cca4e197 admin admin] PATH_INFO: `/v3/users/a125159ca6cb43169e9c336ae9f85f48` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:31.676264 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac2e29a-ef7a-48c9-94e0-d8d6cca4e197 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a125159ca6cb43169e9c336ae9f85f48)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:31.678211 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac2e29a-ef7a-48c9-94e0-d8d6cca4e197 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:31.843435 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-cac2e29a-ef7a-48c9-94e0-d8d6cca4e197 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a125159ca6cb43169e9c336ae9f85f48'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:31.843998 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-cac2e29a-ef7a-48c9-94e0-d8d6cca4e197 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a125159ca6cb43169e9c336ae9f85f48'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:31.844621 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-cac2e29a-ef7a-48c9-94e0-d8d6cca4e197 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a125159ca6cb43169e9c336ae9f85f48'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:31.857372 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-cac2e29a-ef7a-48c9-94e0-d8d6cca4e197 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a125159ca6cb43169e9c336ae9f85f48'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:31.908900 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-cac2e29a-ef7a-48c9-94e0-d8d6cca4e197 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a125159ca6cb43169e9c336ae9f85f48'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:31.922371 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1409/2820] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 09:10:31 2022] DELETE /identity/v3/users/a125159ca6cb43169e9c336ae9f85f48 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:10:31.926900 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f91651-267c-4f38-a0a1-c954c964d813 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:31.927395 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f91651-267c-4f38-a0a1-c954c964d813 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:31.927532 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f91651-267c-4f38-a0a1-c954c964d813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:31.947599 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-d8f91651-267c-4f38-a0a1-c954c964d813 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:10:31.949361 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-d8f91651-267c-4f38-a0a1-c954c964d813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:32.010243 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1412/2821] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:10:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:10:32.021965 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac2e29a-ef7a-48c9-94e0-d8d6cca4e197 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:32.023448 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-cac2e29a-ef7a-48c9-94e0-d8d6cca4e197 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:32.056966 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:10:32.084972 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac2e29a-ef7a-48c9-94e0-d8d6cca4e197 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:32.088841 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9751a60d-c425-47e7-bf39-d6315de11b0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:32.089457 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9751a60d-c425-47e7-bf39-d6315de11b0c service neutron] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:32.089737 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9751a60d-c425-47e7-bf39-d6315de11b0c service neutron] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:32.090024 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9751a60d-c425-47e7-bf39-d6315de11b0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:32.100149 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9751a60d-c425-47e7-bf39-d6315de11b0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:32.101564 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9751a60d-c425-47e7-bf39-d6315de11b0c service neutron] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:32.111524 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1410/2822] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:10:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:10:32.568165 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f91651-267c-4f38-a0a1-c954c964d813 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:32.569785 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-d8f91651-267c-4f38-a0a1-c954c964d813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:32.584284 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f91651-267c-4f38-a0a1-c954c964d813 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:32.590645 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda568cc-99af-4257-b76b-7ed35e3d48b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:32.591405 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda568cc-99af-4257-b76b-7ed35e3d48b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:32.591564 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda568cc-99af-4257-b76b-7ed35e3d48b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:32.591936 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda568cc-99af-4257-b76b-7ed35e3d48b4 admin admin] PATH_INFO: `/v3/projects/b73fa10e2211420bbe0909ad482ab4e6` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:32.593109 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda568cc-99af-4257-b76b-7ed35e3d48b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b73fa10e2211420bbe0909ad482ab4e6)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:32.595015 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda568cc-99af-4257-b76b-7ed35e3d48b4 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:32.716185 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-eda568cc-99af-4257-b76b-7ed35e3d48b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b73fa10e2211420bbe0909ad482ab4e6', 'request_id': 'req-eda568cc-99af-4257-b76b-7ed35e3d48b4'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:32.721764 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-eda568cc-99af-4257-b76b-7ed35e3d48b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b73fa10e2211420bbe0909ad482ab4e6', 'request_id': 'req-eda568cc-99af-4257-b76b-7ed35e3d48b4'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:32.771862 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-eda568cc-99af-4257-b76b-7ed35e3d48b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b73fa10e2211420bbe0909ad482ab4e6', 'request_id': 'req-eda568cc-99af-4257-b76b-7ed35e3d48b4'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:32.784108 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-eda568cc-99af-4257-b76b-7ed35e3d48b4 admin admin] The token cache is being invalidate because project b73fa10e2211420bbe0909ad482ab4e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:10:32.790273 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-eda568cc-99af-4257-b76b-7ed35e3d48b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:32.790273 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-eda568cc-99af-4257-b76b-7ed35e3d48b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:32.790273 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1413/2823] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 09:10:32 2022] DELETE /identity/v3/projects/b73fa10e2211420bbe0909ad482ab4e6 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:10:32.791106 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9751a60d-c425-47e7-bf39-d6315de11b0c service neutron] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:32.800870 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9751a60d-c425-47e7-bf39-d6315de11b0c service neutron] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:32.862358 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d288e9-9aa3-454d-9ab8-5d9189f304dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:32.862996 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d288e9-9aa3-454d-9ab8-5d9189f304dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:32.863280 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d288e9-9aa3-454d-9ab8-5d9189f304dc admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:32.863554 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d288e9-9aa3-454d-9ab8-5d9189f304dc admin admin] PATH_INFO: `/v3/users/74a1290cd2a24063a21013b22238818b` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:32.864774 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d288e9-9aa3-454d-9ab8-5d9189f304dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=74a1290cd2a24063a21013b22238818b)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:32.866721 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d288e9-9aa3-454d-9ab8-5d9189f304dc admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:33.015334 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-a3d288e9-9aa3-454d-9ab8-5d9189f304dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74a1290cd2a24063a21013b22238818b'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:33.016248 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-a3d288e9-9aa3-454d-9ab8-5d9189f304dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74a1290cd2a24063a21013b22238818b'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:33.016979 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-a3d288e9-9aa3-454d-9ab8-5d9189f304dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74a1290cd2a24063a21013b22238818b'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:33.032341 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-a3d288e9-9aa3-454d-9ab8-5d9189f304dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74a1290cd2a24063a21013b22238818b'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:33.097669 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-a3d288e9-9aa3-454d-9ab8-5d9189f304dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74a1290cd2a24063a21013b22238818b'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:33.114209 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1411/2824] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 09:10:32 2022] DELETE /identity/v3/users/74a1290cd2a24063a21013b22238818b => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:10:33.555905 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda568cc-99af-4257-b76b-7ed35e3d48b4 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:33.557949 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-eda568cc-99af-4257-b76b-7ed35e3d48b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:33.574820 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda568cc-99af-4257-b76b-7ed35e3d48b4 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:33.612210 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:10:33.640448 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7455358-ffc7-4be7-b8b3-9210ce237329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:33.641202 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7455358-ffc7-4be7-b8b3-9210ce237329 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:33.641481 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7455358-ffc7-4be7-b8b3-9210ce237329 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:33.641754 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7455358-ffc7-4be7-b8b3-9210ce237329 admin admin] PATH_INFO: `/v3/projects/f656770d7e53463cadc8d95e0e6bb493` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:33.642907 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7455358-ffc7-4be7-b8b3-9210ce237329 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f656770d7e53463cadc8d95e0e6bb493)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:33.644878 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7455358-ffc7-4be7-b8b3-9210ce237329 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:33.776945 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-f7455358-ffc7-4be7-b8b3-9210ce237329 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f656770d7e53463cadc8d95e0e6bb493', 'request_id': 'req-f7455358-ffc7-4be7-b8b3-9210ce237329'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:33.782578 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-f7455358-ffc7-4be7-b8b3-9210ce237329 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f656770d7e53463cadc8d95e0e6bb493', 'request_id': 'req-f7455358-ffc7-4be7-b8b3-9210ce237329'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:33.868716 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-f7455358-ffc7-4be7-b8b3-9210ce237329 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f656770d7e53463cadc8d95e0e6bb493', 'request_id': 'req-f7455358-ffc7-4be7-b8b3-9210ce237329'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:33.881515 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-f7455358-ffc7-4be7-b8b3-9210ce237329 admin admin] The token cache is being invalidate because project f656770d7e53463cadc8d95e0e6bb493 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:10:33.881808 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-f7455358-ffc7-4be7-b8b3-9210ce237329 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:33.882447 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-f7455358-ffc7-4be7-b8b3-9210ce237329 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:33.883914 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1414/2825] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 09:10:33 2022] DELETE /identity/v3/projects/f656770d7e53463cadc8d95e0e6bb493 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:10:33.888917 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887c335f-fa5e-420f-b947-d71445a09782 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:33.889904 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887c335f-fa5e-420f-b947-d71445a09782 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:33.889904 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887c335f-fa5e-420f-b947-d71445a09782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:33.909483 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-887c335f-fa5e-420f-b947-d71445a09782 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:10:33.910721 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-887c335f-fa5e-420f-b947-d71445a09782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:33.955714 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:10:33.971316 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1412/2826] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:10:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:10:33.979271 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1158da1-d77a-4f16-b826-bba54c461a11 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:33.979652 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1158da1-d77a-4f16-b826-bba54c461a11 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:33.979925 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1158da1-d77a-4f16-b826-bba54c461a11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:33.999800 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-c1158da1-d77a-4f16-b826-bba54c461a11 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:10:34.004592 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-c1158da1-d77a-4f16-b826-bba54c461a11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:34.008897 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1415/2827] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:10:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:10:34.013270 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887c335f-fa5e-420f-b947-d71445a09782 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:34.023237 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887c335f-fa5e-420f-b947-d71445a09782 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:34.029142 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709c5746-5d8b-4e6c-ac6b-46c23f6b6b1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:34.029750 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709c5746-5d8b-4e6c-ac6b-46c23f6b6b1d admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:34.030034 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709c5746-5d8b-4e6c-ac6b-46c23f6b6b1d admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:34.030306 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709c5746-5d8b-4e6c-ac6b-46c23f6b6b1d admin admin] PATH_INFO: `/v3/domains` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:34.031009 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709c5746-5d8b-4e6c-ac6b-46c23f6b6b1d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:10:34.031315 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709c5746-5d8b-4e6c-ac6b-46c23f6b6b1d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:34.032958 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709c5746-5d8b-4e6c-ac6b-46c23f6b6b1d admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:34.045477 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1413/2828] 10.222.0.55 () {58 vars in 1153 bytes} [Wed Aug 24 09:10:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:10:34.050074 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1158da1-d77a-4f16-b826-bba54c461a11 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:34.059448 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1158da1-d77a-4f16-b826-bba54c461a11 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:34.064120 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f22fe8-4848-451f-8ff0-4b8beee53e83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:34.067624 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f22fe8-4848-451f-8ff0-4b8beee53e83 admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:34.067624 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f22fe8-4848-451f-8ff0-4b8beee53e83 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:34.067624 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f22fe8-4848-451f-8ff0-4b8beee53e83 admin admin] PATH_INFO: `/v3/projects` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:34.067624 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f22fe8-4848-451f-8ff0-4b8beee53e83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:34.068143 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f22fe8-4848-451f-8ff0-4b8beee53e83 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:34.150753 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1416/2829] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 09:10:34 2022] POST /identity/v3/projects => generated 379 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:10:34.155412 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709c5746-5d8b-4e6c-ac6b-46c23f6b6b1d admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:34.166706 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709c5746-5d8b-4e6c-ac6b-46c23f6b6b1d admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:34.224944 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629b8f8b-4bd7-4541-98a9-899bd022c6df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:34.225549 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629b8f8b-4bd7-4541-98a9-899bd022c6df admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:34.225827 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629b8f8b-4bd7-4541-98a9-899bd022c6df admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:34.226098 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629b8f8b-4bd7-4541-98a9-899bd022c6df admin admin] PATH_INFO: `/v3/users` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:34.226725 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629b8f8b-4bd7-4541-98a9-899bd022c6df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:34.228628 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629b8f8b-4bd7-4541-98a9-899bd022c6df admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:34.320519 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1414/2830] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 09:10:34 2022] POST /identity/v3/users => generated 290 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:10:34.325397 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f22fe8-4848-451f-8ff0-4b8beee53e83 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:34.336007 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f22fe8-4848-451f-8ff0-4b8beee53e83 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:34.341207 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5899f1-559f-4d72-bb41-4b4b30aa3831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:34.341873 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5899f1-559f-4d72-bb41-4b4b30aa3831 admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:34.342144 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5899f1-559f-4d72-bb41-4b4b30aa3831 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:34.342413 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5899f1-559f-4d72-bb41-4b4b30aa3831 admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:34.343045 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5899f1-559f-4d72-bb41-4b4b30aa3831 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:34.344637 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5899f1-559f-4d72-bb41-4b4b30aa3831 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:34.354700 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1417/2831] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:10:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:10:34.358755 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629b8f8b-4bd7-4541-98a9-899bd022c6df admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:34.369791 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629b8f8b-4bd7-4541-98a9-899bd022c6df admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:34.374684 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3e3014-a7bf-4228-ac7b-019c5e792756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:34.375279 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3e3014-a7bf-4228-ac7b-019c5e792756 admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:34.375556 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3e3014-a7bf-4228-ac7b-019c5e792756 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:34.375828 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3e3014-a7bf-4228-ac7b-019c5e792756 admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:34.376464 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3e3014-a7bf-4228-ac7b-019c5e792756 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:34.378087 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3e3014-a7bf-4228-ac7b-019c5e792756 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:34.389200 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1415/2832] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:10:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:10:34.393398 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5899f1-559f-4d72-bb41-4b4b30aa3831 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:34.404141 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5899f1-559f-4d72-bb41-4b4b30aa3831 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:34.408758 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00091752-d2f8-4548-9845-25eea84e8783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:34.409402 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00091752-d2f8-4548-9845-25eea84e8783 admin admin] REQUEST_METHOD: `PUT` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:34.409679 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00091752-d2f8-4548-9845-25eea84e8783 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:34.409975 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00091752-d2f8-4548-9845-25eea84e8783 admin admin] PATH_INFO: `/v3/projects/996b6560ca80467e86957e5d883ec81d/users/d37b28fec4c14cf9953b6d1c25970c0a/roles/b674fd0a592e4e0cada811bba5ceea4d` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:34.425452 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00091752-d2f8-4548-9845-25eea84e8783 admin admin] RBAC: Authorizing `identity:create_grant(project_id=996b6560ca80467e86957e5d883ec81d, user_id=d37b28fec4c14cf9953b6d1c25970c0a, role_id=b674fd0a592e4e0cada811bba5ceea4d)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:34.428088 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00091752-d2f8-4548-9845-25eea84e8783 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:34.532247 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1418/2833] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 09:10:34 2022] PUT /identity/v3/projects/996b6560ca80467e86957e5d883ec81d/users/d37b28fec4c14cf9953b6d1c25970c0a/roles/b674fd0a592e4e0cada811bba5ceea4d => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:10:34.543084 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6069744c-dd63-43f7-8c33-9e8ba666e7da None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:34.543376 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6069744c-dd63-43f7-8c33-9e8ba666e7da None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:34.543651 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6069744c-dd63-43f7-8c33-9e8ba666e7da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:34.572713 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-6069744c-dd63-43f7-8c33-9e8ba666e7da None None] MFA Rules not processed for user `d37b28fec4c14cf9953b6d1c25970c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:10:34.573933 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-6069744c-dd63-43f7-8c33-9e8ba666e7da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:34.640787 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1416/2834] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:10:34 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:10:34.645613 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00091752-d2f8-4548-9845-25eea84e8783 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:34.708553 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00091752-d2f8-4548-9845-25eea84e8783 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:34.713194 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f70654d-c728-464e-a02a-7377c092a838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:34.713783 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f70654d-c728-464e-a02a-7377c092a838 admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:34.714097 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f70654d-c728-464e-a02a-7377c092a838 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:34.714329 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f70654d-c728-464e-a02a-7377c092a838 admin admin] PATH_INFO: `/v3/projects` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:34.714938 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f70654d-c728-464e-a02a-7377c092a838 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:34.717647 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f70654d-c728-464e-a02a-7377c092a838 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:34.722345 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:10:34.801949 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1419/2835] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 09:10:34 2022] POST /identity/v3/projects => generated 381 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:10:34.806234 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6069744c-dd63-43f7-8c33-9e8ba666e7da None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:34.817072 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6069744c-dd63-43f7-8c33-9e8ba666e7da None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:34.859226 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:10:34.873561 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bfb2f7-64bd-4dd7-8108-c536fefbf9ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:34.874185 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bfb2f7-64bd-4dd7-8108-c536fefbf9ed admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:34.874436 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bfb2f7-64bd-4dd7-8108-c536fefbf9ed admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:34.874704 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bfb2f7-64bd-4dd7-8108-c536fefbf9ed admin admin] PATH_INFO: `/v3/users` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:34.875330 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bfb2f7-64bd-4dd7-8108-c536fefbf9ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:34.877260 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bfb2f7-64bd-4dd7-8108-c536fefbf9ed admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:34.937509 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1417/2836] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 09:10:34 2022] POST /identity/v3/users => generated 297 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:10:34.941533 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f70654d-c728-464e-a02a-7377c092a838 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:34.952456 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f70654d-c728-464e-a02a-7377c092a838 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:34.957121 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a930bdd7-9c67-4fba-83dd-a2be0ba151bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:34.957719 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a930bdd7-9c67-4fba-83dd-a2be0ba151bb admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:34.957994 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a930bdd7-9c67-4fba-83dd-a2be0ba151bb admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:34.958263 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a930bdd7-9c67-4fba-83dd-a2be0ba151bb admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:34.958958 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a930bdd7-9c67-4fba-83dd-a2be0ba151bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:34.960517 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a930bdd7-9c67-4fba-83dd-a2be0ba151bb admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:34.970914 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1420/2837] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:10:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:10:34.975290 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bfb2f7-64bd-4dd7-8108-c536fefbf9ed admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:34.985934 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bfb2f7-64bd-4dd7-8108-c536fefbf9ed admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:34.990617 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912b9c90-2501-487a-bb28-e999cc6a0008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:34.991183 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912b9c90-2501-487a-bb28-e999cc6a0008 admin admin] REQUEST_METHOD: `PUT` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:34.991459 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912b9c90-2501-487a-bb28-e999cc6a0008 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:34.991727 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912b9c90-2501-487a-bb28-e999cc6a0008 admin admin] PATH_INFO: `/v3/projects/39e6f9a590e144898966cedced1e900b/users/83b75f2c143341f69d2e4f1423c58acc/roles/b870cd1acaa744e1ae1d58b9ac283821` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:35.006478 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912b9c90-2501-487a-bb28-e999cc6a0008 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39e6f9a590e144898966cedced1e900b, user_id=83b75f2c143341f69d2e4f1423c58acc, role_id=b870cd1acaa744e1ae1d58b9ac283821)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:35.009175 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912b9c90-2501-487a-bb28-e999cc6a0008 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:35.075215 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1418/2838] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 09:10:34 2022] PUT /identity/v3/projects/39e6f9a590e144898966cedced1e900b/users/83b75f2c143341f69d2e4f1423c58acc/roles/b870cd1acaa744e1ae1d58b9ac283821 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:10:35.079300 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a930bdd7-9c67-4fba-83dd-a2be0ba151bb admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:35.089117 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a930bdd7-9c67-4fba-83dd-a2be0ba151bb admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:35.146197 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edd7200-3df8-4b80-966f-7ca22c811ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:35.146800 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd7200-3df8-4b80-966f-7ca22c811ba2 admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:35.147077 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd7200-3df8-4b80-966f-7ca22c811ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:35.147347 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd7200-3df8-4b80-966f-7ca22c811ba2 admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:35.147982 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edd7200-3df8-4b80-966f-7ca22c811ba2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:35.149588 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edd7200-3df8-4b80-966f-7ca22c811ba2 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:35.159933 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1421/2839] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:10:35 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:10:35.164767 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912b9c90-2501-487a-bb28-e999cc6a0008 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:35.175640 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912b9c90-2501-487a-bb28-e999cc6a0008 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:35.181445 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e910f2-09f7-4060-8d87-c529e109715a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:35.182140 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e910f2-09f7-4060-8d87-c529e109715a admin admin] REQUEST_METHOD: `PUT` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:35.182415 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e910f2-09f7-4060-8d87-c529e109715a admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:35.182684 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e910f2-09f7-4060-8d87-c529e109715a admin admin] PATH_INFO: `/v3/projects/39e6f9a590e144898966cedced1e900b/users/83b75f2c143341f69d2e4f1423c58acc/roles/b870cd1acaa744e1ae1d58b9ac283821` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:35.184798 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e910f2-09f7-4060-8d87-c529e109715a admin admin] RBAC: Authorizing `identity:create_grant(project_id=39e6f9a590e144898966cedced1e900b, user_id=83b75f2c143341f69d2e4f1423c58acc, role_id=b870cd1acaa744e1ae1d58b9ac283821)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:35.187487 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e910f2-09f7-4060-8d87-c529e109715a admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:35.200644 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1419/2840] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 09:10:35 2022] PUT /identity/v3/projects/39e6f9a590e144898966cedced1e900b/users/83b75f2c143341f69d2e4f1423c58acc/roles/b870cd1acaa744e1ae1d58b9ac283821 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:10:35.270697 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff926623-4b28-47b8-b0e5-e63a19f1e4d9 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:35.270988 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff926623-4b28-47b8-b0e5-e63a19f1e4d9 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:35.271311 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff926623-4b28-47b8-b0e5-e63a19f1e4d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:35.304212 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-ff926623-4b28-47b8-b0e5-e63a19f1e4d9 None None] MFA Rules not processed for user `83b75f2c143341f69d2e4f1423c58acc`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:10:35.305601 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-ff926623-4b28-47b8-b0e5-e63a19f1e4d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:35.376502 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1422/2841] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:10:35 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:10:35.387961 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e910f2-09f7-4060-8d87-c529e109715a admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:35.389734 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-d0e910f2-09f7-4060-8d87-c529e109715a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:35.447799 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e910f2-09f7-4060-8d87-c529e109715a admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:35.452647 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cc943b-ac1e-4f07-8deb-434ff55357c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbb9ac667f04c35a63b2689df18efed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:35.453197 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cc943b-ac1e-4f07-8deb-434ff55357c6 service cinder] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:35.453475 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cc943b-ac1e-4f07-8deb-434ff55357c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:35.453748 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cc943b-ac1e-4f07-8deb-434ff55357c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:35.462838 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cc943b-ac1e-4f07-8deb-434ff55357c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:35.464272 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cc943b-ac1e-4f07-8deb-434ff55357c6 service cinder] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:35.474426 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1420/2842] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:10:35 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:10:36.014674 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff926623-4b28-47b8-b0e5-e63a19f1e4d9 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:36.072938 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:10:36.078509 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff926623-4b28-47b8-b0e5-e63a19f1e4d9 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:36.082977 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b98ddb4-2bd1-4e3a-91f0-003313928a35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:36.083674 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b98ddb4-2bd1-4e3a-91f0-003313928a35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:36.083842 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b98ddb4-2bd1-4e3a-91f0-003313928a35 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:36.084109 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b98ddb4-2bd1-4e3a-91f0-003313928a35 admin admin] PATH_INFO: `/v3/users/d37b28fec4c14cf9953b6d1c25970c0a` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:36.085230 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b98ddb4-2bd1-4e3a-91f0-003313928a35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d37b28fec4c14cf9953b6d1c25970c0a)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:36.087141 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b98ddb4-2bd1-4e3a-91f0-003313928a35 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:36.243444 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-4b98ddb4-2bd1-4e3a-91f0-003313928a35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd37b28fec4c14cf9953b6d1c25970c0a'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:36.244143 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-4b98ddb4-2bd1-4e3a-91f0-003313928a35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd37b28fec4c14cf9953b6d1c25970c0a'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:36.244719 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-4b98ddb4-2bd1-4e3a-91f0-003313928a35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd37b28fec4c14cf9953b6d1c25970c0a'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:36.256709 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-4b98ddb4-2bd1-4e3a-91f0-003313928a35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd37b28fec4c14cf9953b6d1c25970c0a'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:36.331729 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-4b98ddb4-2bd1-4e3a-91f0-003313928a35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd37b28fec4c14cf9953b6d1c25970c0a'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:36.345173 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1423/2843] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 09:10:36 2022] DELETE /identity/v3/users/d37b28fec4c14cf9953b6d1c25970c0a => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:10:36.349910 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f463ea-17b7-40d3-b1ad-8afe145de9f2 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:36.350434 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f463ea-17b7-40d3-b1ad-8afe145de9f2 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:36.350434 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f463ea-17b7-40d3-b1ad-8afe145de9f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:36.369640 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-84f463ea-17b7-40d3-b1ad-8afe145de9f2 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:10:36.370853 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-84f463ea-17b7-40d3-b1ad-8afe145de9f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:36.428705 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1421/2844] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:10:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:10:36.438461 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b98ddb4-2bd1-4e3a-91f0-003313928a35 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:36.439866 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-4b98ddb4-2bd1-4e3a-91f0-003313928a35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:36.499583 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b98ddb4-2bd1-4e3a-91f0-003313928a35 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:36.503044 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0a600f-d4a2-464b-9932-a331deddf9c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:36.504384 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0a600f-d4a2-464b-9932-a331deddf9c9 service neutron] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:36.504384 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0a600f-d4a2-464b-9932-a331deddf9c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:36.504384 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0a600f-d4a2-464b-9932-a331deddf9c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:36.514178 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0a600f-d4a2-464b-9932-a331deddf9c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:36.515612 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0a600f-d4a2-464b-9932-a331deddf9c9 service neutron] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:36.525483 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1424/2845] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:10:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:10:36.972507 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f463ea-17b7-40d3-b1ad-8afe145de9f2 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:36.974044 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-84f463ea-17b7-40d3-b1ad-8afe145de9f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:36.986450 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f463ea-17b7-40d3-b1ad-8afe145de9f2 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:36.991427 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89aa86f-57e3-4e77-aced-84e9df4ea5e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:36.992091 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89aa86f-57e3-4e77-aced-84e9df4ea5e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:36.992370 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89aa86f-57e3-4e77-aced-84e9df4ea5e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:36.997548 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89aa86f-57e3-4e77-aced-84e9df4ea5e9 admin admin] PATH_INFO: `/v3/projects/996b6560ca80467e86957e5d883ec81d` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:36.997548 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89aa86f-57e3-4e77-aced-84e9df4ea5e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=996b6560ca80467e86957e5d883ec81d)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:36.997548 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89aa86f-57e3-4e77-aced-84e9df4ea5e9 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:37.088106 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:10:37.097585 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-f89aa86f-57e3-4e77-aced-84e9df4ea5e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '996b6560ca80467e86957e5d883ec81d', 'request_id': 'req-f89aa86f-57e3-4e77-aced-84e9df4ea5e9'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:37.165881 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-f89aa86f-57e3-4e77-aced-84e9df4ea5e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '996b6560ca80467e86957e5d883ec81d', 'request_id': 'req-f89aa86f-57e3-4e77-aced-84e9df4ea5e9'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:37.172531 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-f89aa86f-57e3-4e77-aced-84e9df4ea5e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '996b6560ca80467e86957e5d883ec81d', 'request_id': 'req-f89aa86f-57e3-4e77-aced-84e9df4ea5e9'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:37.183455 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-f89aa86f-57e3-4e77-aced-84e9df4ea5e9 admin admin] The token cache is being invalidate because project 996b6560ca80467e86957e5d883ec81d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:10:37.184279 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-f89aa86f-57e3-4e77-aced-84e9df4ea5e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:37.184902 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-f89aa86f-57e3-4e77-aced-84e9df4ea5e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:37.186944 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1422/2846] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 09:10:36 2022] DELETE /identity/v3/projects/996b6560ca80467e86957e5d883ec81d => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:10:37.190969 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0a600f-d4a2-464b-9932-a331deddf9c9 service neutron] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:37.203997 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0a600f-d4a2-464b-9932-a331deddf9c9 service neutron] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:37.258402 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165edb61-5a01-40f7-adcb-8d5d637d56c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:37.258935 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165edb61-5a01-40f7-adcb-8d5d637d56c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:37.259210 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165edb61-5a01-40f7-adcb-8d5d637d56c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:37.259479 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165edb61-5a01-40f7-adcb-8d5d637d56c2 admin admin] PATH_INFO: `/v3/users/83b75f2c143341f69d2e4f1423c58acc` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:37.260617 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165edb61-5a01-40f7-adcb-8d5d637d56c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83b75f2c143341f69d2e4f1423c58acc)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:37.262552 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165edb61-5a01-40f7-adcb-8d5d637d56c2 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:37.283247 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:10:37.436905 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-165edb61-5a01-40f7-adcb-8d5d637d56c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83b75f2c143341f69d2e4f1423c58acc'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:37.437559 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-165edb61-5a01-40f7-adcb-8d5d637d56c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83b75f2c143341f69d2e4f1423c58acc'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:37.438164 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-165edb61-5a01-40f7-adcb-8d5d637d56c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83b75f2c143341f69d2e4f1423c58acc'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:37.450331 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-165edb61-5a01-40f7-adcb-8d5d637d56c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83b75f2c143341f69d2e4f1423c58acc'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:37.508752 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-165edb61-5a01-40f7-adcb-8d5d637d56c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83b75f2c143341f69d2e4f1423c58acc'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:37.521606 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1425/2847] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 09:10:37 2022] DELETE /identity/v3/users/83b75f2c143341f69d2e4f1423c58acc => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:10:37.939491 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89aa86f-57e3-4e77-aced-84e9df4ea5e9 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:37.941078 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-f89aa86f-57e3-4e77-aced-84e9df4ea5e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:37.956082 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89aa86f-57e3-4e77-aced-84e9df4ea5e9 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:38.014198 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285dba74-cab7-4a2d-a266-dd8bce60ec10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:38.014826 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285dba74-cab7-4a2d-a266-dd8bce60ec10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:38.015105 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285dba74-cab7-4a2d-a266-dd8bce60ec10 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:38.015377 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285dba74-cab7-4a2d-a266-dd8bce60ec10 admin admin] PATH_INFO: `/v3/projects/39e6f9a590e144898966cedced1e900b` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:38.016513 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285dba74-cab7-4a2d-a266-dd8bce60ec10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=39e6f9a590e144898966cedced1e900b)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:38.018435 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285dba74-cab7-4a2d-a266-dd8bce60ec10 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:38.125854 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-285dba74-cab7-4a2d-a266-dd8bce60ec10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39e6f9a590e144898966cedced1e900b', 'request_id': 'req-285dba74-cab7-4a2d-a266-dd8bce60ec10'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:38.211672 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-285dba74-cab7-4a2d-a266-dd8bce60ec10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39e6f9a590e144898966cedced1e900b', 'request_id': 'req-285dba74-cab7-4a2d-a266-dd8bce60ec10'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:38.219504 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-285dba74-cab7-4a2d-a266-dd8bce60ec10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39e6f9a590e144898966cedced1e900b', 'request_id': 'req-285dba74-cab7-4a2d-a266-dd8bce60ec10'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:38.230056 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-285dba74-cab7-4a2d-a266-dd8bce60ec10 admin admin] The token cache is being invalidate because project 39e6f9a590e144898966cedced1e900b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:10:38.231032 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-285dba74-cab7-4a2d-a266-dd8bce60ec10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:38.231774 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-285dba74-cab7-4a2d-a266-dd8bce60ec10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:38.233426 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1423/2848] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 09:10:37 2022] DELETE /identity/v3/projects/39e6f9a590e144898966cedced1e900b => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:10:38.238329 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da032578-7b5d-4439-9874-b1b53760e035 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:38.238844 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da032578-7b5d-4439-9874-b1b53760e035 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:38.239956 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da032578-7b5d-4439-9874-b1b53760e035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:38.260833 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-da032578-7b5d-4439-9874-b1b53760e035 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:10:38.262038 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-da032578-7b5d-4439-9874-b1b53760e035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:38.322490 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1426/2849] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:10:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:10:38.330706 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5b9fa1-0b69-4b96-8832-6a611e95fb45 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:38.330994 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5b9fa1-0b69-4b96-8832-6a611e95fb45 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:38.331270 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5b9fa1-0b69-4b96-8832-6a611e95fb45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:38.352594 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-de5b9fa1-0b69-4b96-8832-6a611e95fb45 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:10:38.353835 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-de5b9fa1-0b69-4b96-8832-6a611e95fb45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:38.361880 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1424/2850] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:10:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:10:38.366598 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da032578-7b5d-4439-9874-b1b53760e035 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:38.376968 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da032578-7b5d-4439-9874-b1b53760e035 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:38.381457 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8190823-b56d-4e8e-ae3d-9f239bf3af44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:38.382050 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8190823-b56d-4e8e-ae3d-9f239bf3af44 admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:38.382321 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8190823-b56d-4e8e-ae3d-9f239bf3af44 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:38.382609 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8190823-b56d-4e8e-ae3d-9f239bf3af44 admin admin] PATH_INFO: `/v3/domains` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:38.383224 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8190823-b56d-4e8e-ae3d-9f239bf3af44 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:10:38.383515 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8190823-b56d-4e8e-ae3d-9f239bf3af44 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:38.385155 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8190823-b56d-4e8e-ae3d-9f239bf3af44 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:38.398387 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1427/2851] 10.222.0.55 () {58 vars in 1153 bytes} [Wed Aug 24 09:10:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:10:38.403091 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5b9fa1-0b69-4b96-8832-6a611e95fb45 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:38.412282 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5b9fa1-0b69-4b96-8832-6a611e95fb45 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:38.417291 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c418f24a-3d9d-4733-a845-176da1826f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:38.417880 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c418f24a-3d9d-4733-a845-176da1826f2e admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:38.418155 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c418f24a-3d9d-4733-a845-176da1826f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:38.418429 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c418f24a-3d9d-4733-a845-176da1826f2e admin admin] PATH_INFO: `/v3/projects` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:38.419042 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c418f24a-3d9d-4733-a845-176da1826f2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:38.422103 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c418f24a-3d9d-4733-a845-176da1826f2e admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:38.522710 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1425/2852] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 09:10:38 2022] POST /identity/v3/projects => generated 373 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:10:38.527025 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8190823-b56d-4e8e-ae3d-9f239bf3af44 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:38.537262 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8190823-b56d-4e8e-ae3d-9f239bf3af44 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:38.594892 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75cab5c-43ac-4957-8d9c-710053ff461c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:38.595501 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75cab5c-43ac-4957-8d9c-710053ff461c admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:38.595780 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75cab5c-43ac-4957-8d9c-710053ff461c admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:38.596052 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75cab5c-43ac-4957-8d9c-710053ff461c admin admin] PATH_INFO: `/v3/users` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:38.596719 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75cab5c-43ac-4957-8d9c-710053ff461c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:38.598619 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75cab5c-43ac-4957-8d9c-710053ff461c admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:38.703432 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1428/2853] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 09:10:38 2022] POST /identity/v3/users => generated 287 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:10:38.707676 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c418f24a-3d9d-4733-a845-176da1826f2e admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:38.718101 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c418f24a-3d9d-4733-a845-176da1826f2e admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:38.722813 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9d3c50-e404-4834-9afb-51e5800e11d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:38.723398 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9d3c50-e404-4834-9afb-51e5800e11d5 admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:38.723673 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9d3c50-e404-4834-9afb-51e5800e11d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:38.723943 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9d3c50-e404-4834-9afb-51e5800e11d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:38.724605 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9d3c50-e404-4834-9afb-51e5800e11d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:38.726164 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9d3c50-e404-4834-9afb-51e5800e11d5 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:38.736579 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1426/2854] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:10:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:10:38.740603 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75cab5c-43ac-4957-8d9c-710053ff461c admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:38.751312 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75cab5c-43ac-4957-8d9c-710053ff461c admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:38.756142 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c64913-f9f8-4979-9385-735f0a1ecc45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:38.756780 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c64913-f9f8-4979-9385-735f0a1ecc45 admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:38.757060 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c64913-f9f8-4979-9385-735f0a1ecc45 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:38.757332 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c64913-f9f8-4979-9385-735f0a1ecc45 admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:38.757970 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c64913-f9f8-4979-9385-735f0a1ecc45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:38.759551 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c64913-f9f8-4979-9385-735f0a1ecc45 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:38.769940 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1429/2855] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:10:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:10:38.774370 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9d3c50-e404-4834-9afb-51e5800e11d5 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:38.784301 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9d3c50-e404-4834-9afb-51e5800e11d5 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:38.789100 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4496cfd-3087-42af-a92e-ddb689a7c5d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:38.789737 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4496cfd-3087-42af-a92e-ddb689a7c5d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:38.790009 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4496cfd-3087-42af-a92e-ddb689a7c5d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:38.790277 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4496cfd-3087-42af-a92e-ddb689a7c5d6 admin admin] PATH_INFO: `/v3/projects/0d1981a771c2476cbed8f8e27272a7d9/users/7f318ca18ccb4ac6a2e6ce41bb6abed7/roles/b674fd0a592e4e0cada811bba5ceea4d` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:38.801810 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:10:38.805695 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4496cfd-3087-42af-a92e-ddb689a7c5d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d1981a771c2476cbed8f8e27272a7d9, user_id=7f318ca18ccb4ac6a2e6ce41bb6abed7, role_id=b674fd0a592e4e0cada811bba5ceea4d)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:38.808438 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4496cfd-3087-42af-a92e-ddb689a7c5d6 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:38.920556 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1427/2856] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 09:10:38 2022] PUT /identity/v3/projects/0d1981a771c2476cbed8f8e27272a7d9/users/7f318ca18ccb4ac6a2e6ce41bb6abed7/roles/b674fd0a592e4e0cada811bba5ceea4d => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:10:38.932012 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6daa63-fb00-44ce-9827-5f2adbec40ea None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:38.932295 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6daa63-fb00-44ce-9827-5f2adbec40ea None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:38.932609 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6daa63-fb00-44ce-9827-5f2adbec40ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:38.939756 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:10:38.961726 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-ee6daa63-fb00-44ce-9827-5f2adbec40ea None None] MFA Rules not processed for user `7f318ca18ccb4ac6a2e6ce41bb6abed7`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:10:38.962900 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-ee6daa63-fb00-44ce-9827-5f2adbec40ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:39.027503 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1430/2857] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:10:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:10:39.032334 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4496cfd-3087-42af-a92e-ddb689a7c5d6 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:39.093838 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4496cfd-3087-42af-a92e-ddb689a7c5d6 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:39.099218 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f834eb8-79f1-48a5-96ed-ea7ae29eba87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:39.099826 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f834eb8-79f1-48a5-96ed-ea7ae29eba87 admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:39.100103 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f834eb8-79f1-48a5-96ed-ea7ae29eba87 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:39.100372 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f834eb8-79f1-48a5-96ed-ea7ae29eba87 admin admin] PATH_INFO: `/v3/projects` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:39.101035 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f834eb8-79f1-48a5-96ed-ea7ae29eba87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:39.102991 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f834eb8-79f1-48a5-96ed-ea7ae29eba87 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:39.205624 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1428/2858] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 09:10:39 2022] POST /identity/v3/projects => generated 375 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:10:39.206771 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6daa63-fb00-44ce-9827-5f2adbec40ea None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:39.217691 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6daa63-fb00-44ce-9827-5f2adbec40ea None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:39.274281 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca51341-563b-4c7c-9b7f-f1bcabd34ea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:39.274955 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca51341-563b-4c7c-9b7f-f1bcabd34ea8 admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:39.275148 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca51341-563b-4c7c-9b7f-f1bcabd34ea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:39.275423 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca51341-563b-4c7c-9b7f-f1bcabd34ea8 admin admin] PATH_INFO: `/v3/users` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:39.276037 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca51341-563b-4c7c-9b7f-f1bcabd34ea8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:39.278006 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca51341-563b-4c7c-9b7f-f1bcabd34ea8 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:39.406099 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1431/2859] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 09:10:39 2022] POST /identity/v3/users => generated 294 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:10:39.410175 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f834eb8-79f1-48a5-96ed-ea7ae29eba87 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:39.421625 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f834eb8-79f1-48a5-96ed-ea7ae29eba87 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:39.426350 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd86865-2984-4c23-840f-8883558dc35b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:39.426936 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd86865-2984-4c23-840f-8883558dc35b admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:39.427216 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd86865-2984-4c23-840f-8883558dc35b admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:39.427507 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd86865-2984-4c23-840f-8883558dc35b admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:39.428149 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd86865-2984-4c23-840f-8883558dc35b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:39.429778 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd86865-2984-4c23-840f-8883558dc35b admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:39.440862 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1429/2860] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:10:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:10:39.445285 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca51341-563b-4c7c-9b7f-f1bcabd34ea8 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:39.455121 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca51341-563b-4c7c-9b7f-f1bcabd34ea8 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:39.459676 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b3edcd-24f1-447a-ac18-a1df30fd909b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:39.460300 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b3edcd-24f1-447a-ac18-a1df30fd909b admin admin] REQUEST_METHOD: `PUT` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:39.460607 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b3edcd-24f1-447a-ac18-a1df30fd909b admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:39.460879 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b3edcd-24f1-447a-ac18-a1df30fd909b admin admin] PATH_INFO: `/v3/projects/85a3ad41d48c45eab4d186fb6f1df500/users/09b61ef6c20047a6833cdcd286d686b0/roles/b870cd1acaa744e1ae1d58b9ac283821` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:39.475351 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b3edcd-24f1-447a-ac18-a1df30fd909b admin admin] RBAC: Authorizing `identity:create_grant(project_id=85a3ad41d48c45eab4d186fb6f1df500, user_id=09b61ef6c20047a6833cdcd286d686b0, role_id=b870cd1acaa744e1ae1d58b9ac283821)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:39.477948 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b3edcd-24f1-447a-ac18-a1df30fd909b admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:39.550143 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1432/2861] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 09:10:39 2022] PUT /identity/v3/projects/85a3ad41d48c45eab4d186fb6f1df500/users/09b61ef6c20047a6833cdcd286d686b0/roles/b870cd1acaa744e1ae1d58b9ac283821 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:10:39.554058 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd86865-2984-4c23-840f-8883558dc35b admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:39.565065 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd86865-2984-4c23-840f-8883558dc35b admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:39.620518 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6a02b3-e326-46d2-b543-f07d6646a859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:39.621153 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6a02b3-e326-46d2-b543-f07d6646a859 admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:39.621436 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6a02b3-e326-46d2-b543-f07d6646a859 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:39.621715 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6a02b3-e326-46d2-b543-f07d6646a859 admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:39.622355 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6a02b3-e326-46d2-b543-f07d6646a859 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:39.624746 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6a02b3-e326-46d2-b543-f07d6646a859 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:39.635709 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1430/2862] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:10:39 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:10:39.639989 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b3edcd-24f1-447a-ac18-a1df30fd909b admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:39.650348 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b3edcd-24f1-447a-ac18-a1df30fd909b admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:39.655063 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809311eb-3050-4224-98c5-3515c20f6655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:39.655736 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809311eb-3050-4224-98c5-3515c20f6655 admin admin] REQUEST_METHOD: `PUT` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:39.656015 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809311eb-3050-4224-98c5-3515c20f6655 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:39.656291 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809311eb-3050-4224-98c5-3515c20f6655 admin admin] PATH_INFO: `/v3/projects/85a3ad41d48c45eab4d186fb6f1df500/users/09b61ef6c20047a6833cdcd286d686b0/roles/b870cd1acaa744e1ae1d58b9ac283821` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:39.658538 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809311eb-3050-4224-98c5-3515c20f6655 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85a3ad41d48c45eab4d186fb6f1df500, user_id=09b61ef6c20047a6833cdcd286d686b0, role_id=b870cd1acaa744e1ae1d58b9ac283821)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:39.661212 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809311eb-3050-4224-98c5-3515c20f6655 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:39.672343 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1433/2863] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 09:10:39 2022] PUT /identity/v3/projects/85a3ad41d48c45eab4d186fb6f1df500/users/09b61ef6c20047a6833cdcd286d686b0/roles/b870cd1acaa744e1ae1d58b9ac283821 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:10:39.683023 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7cb547-6208-4bb7-a694-c9a93a7e5d6b None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:39.683308 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7cb547-6208-4bb7-a694-c9a93a7e5d6b None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:39.683585 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7cb547-6208-4bb7-a694-c9a93a7e5d6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:39.711246 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-db7cb547-6208-4bb7-a694-c9a93a7e5d6b None None] MFA Rules not processed for user `09b61ef6c20047a6833cdcd286d686b0`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:10:39.712456 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-db7cb547-6208-4bb7-a694-c9a93a7e5d6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:39.758469 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:10:39.779312 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1431/2864] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:10:39 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:10:39.790867 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809311eb-3050-4224-98c5-3515c20f6655 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:39.792584 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-809311eb-3050-4224-98c5-3515c20f6655 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:39.839599 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:10:39.850752 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809311eb-3050-4224-98c5-3515c20f6655 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:39.854432 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220ee106-a28e-414b-8c2b-420f778334bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbb9ac667f04c35a63b2689df18efed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:39.855031 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220ee106-a28e-414b-8c2b-420f778334bb service cinder] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:39.855309 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220ee106-a28e-414b-8c2b-420f778334bb service cinder] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:39.855577 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220ee106-a28e-414b-8c2b-420f778334bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:39.913776 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220ee106-a28e-414b-8c2b-420f778334bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:39.915506 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220ee106-a28e-414b-8c2b-420f778334bb service cinder] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:39.925580 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1434/2865] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:10:39 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:10:44.404364 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7cb547-6208-4bb7-a694-c9a93a7e5d6b None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:44.466782 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7cb547-6208-4bb7-a694-c9a93a7e5d6b None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:44.471357 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4af1a29-1288-4587-b19e-4ae8f93290d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:44.471979 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4af1a29-1288-4587-b19e-4ae8f93290d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:44.472257 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4af1a29-1288-4587-b19e-4ae8f93290d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:44.472527 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4af1a29-1288-4587-b19e-4ae8f93290d6 admin admin] PATH_INFO: `/v3/users/7f318ca18ccb4ac6a2e6ce41bb6abed7` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:44.473674 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4af1a29-1288-4587-b19e-4ae8f93290d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f318ca18ccb4ac6a2e6ce41bb6abed7)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:44.475581 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4af1a29-1288-4587-b19e-4ae8f93290d6 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:44.820078 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-c4af1a29-1288-4587-b19e-4ae8f93290d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f318ca18ccb4ac6a2e6ce41bb6abed7'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:44.820829 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-c4af1a29-1288-4587-b19e-4ae8f93290d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f318ca18ccb4ac6a2e6ce41bb6abed7'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:44.821393 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-c4af1a29-1288-4587-b19e-4ae8f93290d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f318ca18ccb4ac6a2e6ce41bb6abed7'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:44.833208 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-c4af1a29-1288-4587-b19e-4ae8f93290d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f318ca18ccb4ac6a2e6ce41bb6abed7'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:44.931479 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-c4af1a29-1288-4587-b19e-4ae8f93290d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f318ca18ccb4ac6a2e6ce41bb6abed7'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:44.944698 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1432/2866] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 09:10:44 2022] DELETE /identity/v3/users/7f318ca18ccb4ac6a2e6ce41bb6abed7 => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:10:44.949354 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809e2cce-be9b-4a56-ba63-aeaa85e67062 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:44.949843 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809e2cce-be9b-4a56-ba63-aeaa85e67062 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:44.949962 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809e2cce-be9b-4a56-ba63-aeaa85e67062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:44.968380 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-809e2cce-be9b-4a56-ba63-aeaa85e67062 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:10:44.969606 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-809e2cce-be9b-4a56-ba63-aeaa85e67062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:45.029193 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1435/2867] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:10:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:10:45.040213 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4af1a29-1288-4587-b19e-4ae8f93290d6 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:45.041673 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-c4af1a29-1288-4587-b19e-4ae8f93290d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:45.100559 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4af1a29-1288-4587-b19e-4ae8f93290d6 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:45.104255 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115e8e75-518b-45a9-99fe-9c35f161aa1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:45.104905 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115e8e75-518b-45a9-99fe-9c35f161aa1e service neutron] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:45.105184 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115e8e75-518b-45a9-99fe-9c35f161aa1e service neutron] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:45.105480 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115e8e75-518b-45a9-99fe-9c35f161aa1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:45.114988 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115e8e75-518b-45a9-99fe-9c35f161aa1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:45.116405 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115e8e75-518b-45a9-99fe-9c35f161aa1e service neutron] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:45.125782 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1433/2868] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:10:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 09:10:45.590836 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809e2cce-be9b-4a56-ba63-aeaa85e67062 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:45.592449 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-809e2cce-be9b-4a56-ba63-aeaa85e67062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:45.605958 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809e2cce-be9b-4a56-ba63-aeaa85e67062 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:45.610797 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d4dc04-9d7c-4e30-8ea5-4792087f0d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:45.611391 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d4dc04-9d7c-4e30-8ea5-4792087f0d7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:45.611645 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d4dc04-9d7c-4e30-8ea5-4792087f0d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:45.611916 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d4dc04-9d7c-4e30-8ea5-4792087f0d7b admin admin] PATH_INFO: `/v3/projects/0d1981a771c2476cbed8f8e27272a7d9` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:45.613077 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d4dc04-9d7c-4e30-8ea5-4792087f0d7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d1981a771c2476cbed8f8e27272a7d9)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:45.614994 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d4dc04-9d7c-4e30-8ea5-4792087f0d7b admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:45.743602 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-00d4dc04-9d7c-4e30-8ea5-4792087f0d7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d1981a771c2476cbed8f8e27272a7d9', 'request_id': 'req-00d4dc04-9d7c-4e30-8ea5-4792087f0d7b'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:45.749878 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-00d4dc04-9d7c-4e30-8ea5-4792087f0d7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d1981a771c2476cbed8f8e27272a7d9', 'request_id': 'req-00d4dc04-9d7c-4e30-8ea5-4792087f0d7b'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:45.840626 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-00d4dc04-9d7c-4e30-8ea5-4792087f0d7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d1981a771c2476cbed8f8e27272a7d9', 'request_id': 'req-00d4dc04-9d7c-4e30-8ea5-4792087f0d7b'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:45.852594 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-00d4dc04-9d7c-4e30-8ea5-4792087f0d7b admin admin] The token cache is being invalidate because project 0d1981a771c2476cbed8f8e27272a7d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:10:45.852882 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-00d4dc04-9d7c-4e30-8ea5-4792087f0d7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:45.853522 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-00d4dc04-9d7c-4e30-8ea5-4792087f0d7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:45.855054 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1436/2869] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 09:10:45 2022] DELETE /identity/v3/projects/0d1981a771c2476cbed8f8e27272a7d9 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:10:45.859265 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115e8e75-518b-45a9-99fe-9c35f161aa1e service neutron] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:45.869924 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115e8e75-518b-45a9-99fe-9c35f161aa1e service neutron] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:45.874216 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:10:45.927906 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618f4162-33ea-4b71-abd8-477a55ed4511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:45.928526 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618f4162-33ea-4b71-abd8-477a55ed4511 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:45.928832 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618f4162-33ea-4b71-abd8-477a55ed4511 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:45.929121 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618f4162-33ea-4b71-abd8-477a55ed4511 admin admin] PATH_INFO: `/v3/users/09b61ef6c20047a6833cdcd286d686b0` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:45.930330 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618f4162-33ea-4b71-abd8-477a55ed4511 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09b61ef6c20047a6833cdcd286d686b0)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:45.932229 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618f4162-33ea-4b71-abd8-477a55ed4511 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:46.193209 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-618f4162-33ea-4b71-abd8-477a55ed4511 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09b61ef6c20047a6833cdcd286d686b0'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:46.197652 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-618f4162-33ea-4b71-abd8-477a55ed4511 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09b61ef6c20047a6833cdcd286d686b0'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:46.198420 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-618f4162-33ea-4b71-abd8-477a55ed4511 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09b61ef6c20047a6833cdcd286d686b0'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:46.219034 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-618f4162-33ea-4b71-abd8-477a55ed4511 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09b61ef6c20047a6833cdcd286d686b0'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:46.293242 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-618f4162-33ea-4b71-abd8-477a55ed4511 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09b61ef6c20047a6833cdcd286d686b0'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:46.323841 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1434/2870] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 09:10:45 2022] DELETE /identity/v3/users/09b61ef6c20047a6833cdcd286d686b0 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:10:46.865228 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d4dc04-9d7c-4e30-8ea5-4792087f0d7b admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:46.866744 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-00d4dc04-9d7c-4e30-8ea5-4792087f0d7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:46.881932 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d4dc04-9d7c-4e30-8ea5-4792087f0d7b admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:46.903572 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:10:46.947307 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13eecdb-c0ac-4cef-a9b3-102446b1e223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:46.948202 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13eecdb-c0ac-4cef-a9b3-102446b1e223 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:46.948202 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13eecdb-c0ac-4cef-a9b3-102446b1e223 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:46.948444 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13eecdb-c0ac-4cef-a9b3-102446b1e223 admin admin] PATH_INFO: `/v3/projects/85a3ad41d48c45eab4d186fb6f1df500` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:46.949570 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13eecdb-c0ac-4cef-a9b3-102446b1e223 admin admin] RBAC: Authorizing `identity:delete_project(project_id=85a3ad41d48c45eab4d186fb6f1df500)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:46.954165 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13eecdb-c0ac-4cef-a9b3-102446b1e223 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:47.086992 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-f13eecdb-c0ac-4cef-a9b3-102446b1e223 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85a3ad41d48c45eab4d186fb6f1df500', 'request_id': 'req-f13eecdb-c0ac-4cef-a9b3-102446b1e223'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:47.092563 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-f13eecdb-c0ac-4cef-a9b3-102446b1e223 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85a3ad41d48c45eab4d186fb6f1df500', 'request_id': 'req-f13eecdb-c0ac-4cef-a9b3-102446b1e223'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:47.165866 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-f13eecdb-c0ac-4cef-a9b3-102446b1e223 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85a3ad41d48c45eab4d186fb6f1df500', 'request_id': 'req-f13eecdb-c0ac-4cef-a9b3-102446b1e223'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:47.178333 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-f13eecdb-c0ac-4cef-a9b3-102446b1e223 admin admin] The token cache is being invalidate because project 85a3ad41d48c45eab4d186fb6f1df500 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:10:47.178618 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-f13eecdb-c0ac-4cef-a9b3-102446b1e223 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:47.179267 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-f13eecdb-c0ac-4cef-a9b3-102446b1e223 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:47.180765 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1437/2871] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 09:10:46 2022] DELETE /identity/v3/projects/85a3ad41d48c45eab4d186fb6f1df500 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:10:47.185980 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2d77a7-e4ae-4203-8959-38bdcd864182 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:47.186485 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2d77a7-e4ae-4203-8959-38bdcd864182 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:47.186485 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2d77a7-e4ae-4203-8959-38bdcd864182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:47.210072 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-2b2d77a7-e4ae-4203-8959-38bdcd864182 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:10:47.211292 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-2b2d77a7-e4ae-4203-8959-38bdcd864182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:47.271340 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1435/2872] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:10:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:10:47.279332 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92e2b2d-3ed9-415f-854d-10bbf9542da2 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:47.279621 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92e2b2d-3ed9-415f-854d-10bbf9542da2 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:47.279901 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92e2b2d-3ed9-415f-854d-10bbf9542da2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:47.299646 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-b92e2b2d-3ed9-415f-854d-10bbf9542da2 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:10:47.302137 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-b92e2b2d-3ed9-415f-854d-10bbf9542da2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:47.310179 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1438/2873] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:10:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:10:47.315131 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2d77a7-e4ae-4203-8959-38bdcd864182 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:47.324848 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2d77a7-e4ae-4203-8959-38bdcd864182 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:47.329382 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac34fc5-b916-4c06-9aac-6fd86dc82733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:47.330546 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac34fc5-b916-4c06-9aac-6fd86dc82733 admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:47.330819 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac34fc5-b916-4c06-9aac-6fd86dc82733 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:47.331086 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac34fc5-b916-4c06-9aac-6fd86dc82733 admin admin] PATH_INFO: `/v3/domains` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:47.331748 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac34fc5-b916-4c06-9aac-6fd86dc82733 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:10:47.332045 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac34fc5-b916-4c06-9aac-6fd86dc82733 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:47.333662 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac34fc5-b916-4c06-9aac-6fd86dc82733 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:47.345021 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1436/2874] 10.222.0.55 () {58 vars in 1153 bytes} [Wed Aug 24 09:10:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:10:47.349462 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92e2b2d-3ed9-415f-854d-10bbf9542da2 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:47.358755 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92e2b2d-3ed9-415f-854d-10bbf9542da2 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:47.363439 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a90701b-db32-49f3-beaa-739aac200bcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:47.364025 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a90701b-db32-49f3-beaa-739aac200bcf admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:47.364301 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a90701b-db32-49f3-beaa-739aac200bcf admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:47.364609 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a90701b-db32-49f3-beaa-739aac200bcf admin admin] PATH_INFO: `/v3/projects` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:47.365227 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a90701b-db32-49f3-beaa-739aac200bcf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:47.367179 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a90701b-db32-49f3-beaa-739aac200bcf admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:47.459606 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1439/2875] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 09:10:47 2022] POST /identity/v3/projects => generated 371 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:10:47.463685 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac34fc5-b916-4c06-9aac-6fd86dc82733 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:47.473458 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac34fc5-b916-4c06-9aac-6fd86dc82733 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:47.482141 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:10:47.529042 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e8c3f4-6edd-4103-9b80-79fde54af879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:47.529648 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e8c3f4-6edd-4103-9b80-79fde54af879 admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:47.529927 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e8c3f4-6edd-4103-9b80-79fde54af879 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:47.530213 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e8c3f4-6edd-4103-9b80-79fde54af879 admin admin] PATH_INFO: `/v3/users` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:47.530838 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e8c3f4-6edd-4103-9b80-79fde54af879 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:47.532789 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e8c3f4-6edd-4103-9b80-79fde54af879 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:47.657860 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1437/2876] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 09:10:47 2022] POST /identity/v3/users => generated 286 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:10:47.661999 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a90701b-db32-49f3-beaa-739aac200bcf admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:47.673194 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a90701b-db32-49f3-beaa-739aac200bcf admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:47.678411 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d68ed17-c699-457f-b962-74cce50fd9b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:47.679030 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d68ed17-c699-457f-b962-74cce50fd9b5 admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:47.679306 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d68ed17-c699-457f-b962-74cce50fd9b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:47.679574 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d68ed17-c699-457f-b962-74cce50fd9b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:47.680208 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d68ed17-c699-457f-b962-74cce50fd9b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:47.681816 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d68ed17-c699-457f-b962-74cce50fd9b5 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:47.692694 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1440/2877] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:10:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 09:10:47.696860 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e8c3f4-6edd-4103-9b80-79fde54af879 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:47.707410 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e8c3f4-6edd-4103-9b80-79fde54af879 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:47.713875 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b85c4da-906e-4e03-a534-7cf0b50a5bd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:47.713875 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b85c4da-906e-4e03-a534-7cf0b50a5bd3 admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:47.713875 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b85c4da-906e-4e03-a534-7cf0b50a5bd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:47.713875 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b85c4da-906e-4e03-a534-7cf0b50a5bd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:47.714324 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b85c4da-906e-4e03-a534-7cf0b50a5bd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:47.715536 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b85c4da-906e-4e03-a534-7cf0b50a5bd3 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:47.726142 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1438/2878] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:10:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 09:10:47.730487 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d68ed17-c699-457f-b962-74cce50fd9b5 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:47.740931 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d68ed17-c699-457f-b962-74cce50fd9b5 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:47.745406 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d26f9f-d7a3-451d-8e97-97767220d593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:47.746047 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d26f9f-d7a3-451d-8e97-97767220d593 admin admin] REQUEST_METHOD: `PUT` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:47.746323 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d26f9f-d7a3-451d-8e97-97767220d593 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:47.746594 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d26f9f-d7a3-451d-8e97-97767220d593 admin admin] PATH_INFO: `/v3/projects/5cea599a5b274d17b22d275dc246bb57/users/ad886cea71c243eda754e50b8e4d8a92/roles/b674fd0a592e4e0cada811bba5ceea4d` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:47.762632 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d26f9f-d7a3-451d-8e97-97767220d593 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cea599a5b274d17b22d275dc246bb57, user_id=ad886cea71c243eda754e50b8e4d8a92, role_id=b674fd0a592e4e0cada811bba5ceea4d)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:47.765293 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d26f9f-d7a3-451d-8e97-97767220d593 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:47.834246 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1441/2879] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 09:10:47 2022] PUT /identity/v3/projects/5cea599a5b274d17b22d275dc246bb57/users/ad886cea71c243eda754e50b8e4d8a92/roles/b674fd0a592e4e0cada811bba5ceea4d => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:10:47.846036 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820d0c95-d50f-477b-97d1-3a317fb5cdb2 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:47.846320 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820d0c95-d50f-477b-97d1-3a317fb5cdb2 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:47.846593 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820d0c95-d50f-477b-97d1-3a317fb5cdb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:47.874086 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-820d0c95-d50f-477b-97d1-3a317fb5cdb2 None None] MFA Rules not processed for user `ad886cea71c243eda754e50b8e4d8a92`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:10:47.875297 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-820d0c95-d50f-477b-97d1-3a317fb5cdb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:47.940654 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1439/2880] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:10:47 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:10:47.952229 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d26f9f-d7a3-451d-8e97-97767220d593 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:47.953973 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-81d26f9f-d7a3-451d-8e97-97767220d593 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:48.008809 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:10:48.012791 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d26f9f-d7a3-451d-8e97-97767220d593 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:48.017899 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9b7f3c-b7b9-43f5-ae08-15227d3fc35a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbb9ac667f04c35a63b2689df18efed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:48.017899 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9b7f3c-b7b9-43f5-ae08-15227d3fc35a service cinder] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:48.017899 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9b7f3c-b7b9-43f5-ae08-15227d3fc35a service cinder] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:48.017899 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9b7f3c-b7b9-43f5-ae08-15227d3fc35a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:48.026399 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9b7f3c-b7b9-43f5-ae08-15227d3fc35a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:48.028291 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9b7f3c-b7b9-43f5-ae08-15227d3fc35a service cinder] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:48.038315 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1442/2881] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:10:47 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:10:48.064490 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820d0c95-d50f-477b-97d1-3a317fb5cdb2 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:48.129030 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820d0c95-d50f-477b-97d1-3a317fb5cdb2 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:48.134979 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c812c8-de17-4323-a7b7-4cd069da5b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:48.135903 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c812c8-de17-4323-a7b7-4cd069da5b02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:48.136185 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c812c8-de17-4323-a7b7-4cd069da5b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:48.136454 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c812c8-de17-4323-a7b7-4cd069da5b02 admin admin] PATH_INFO: `/v3/users/ad886cea71c243eda754e50b8e4d8a92` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:48.137807 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c812c8-de17-4323-a7b7-4cd069da5b02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad886cea71c243eda754e50b8e4d8a92)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:48.139777 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c812c8-de17-4323-a7b7-4cd069da5b02 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:48.165275 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:10:48.350992 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-a5c812c8-de17-4323-a7b7-4cd069da5b02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad886cea71c243eda754e50b8e4d8a92'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:48.351652 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-a5c812c8-de17-4323-a7b7-4cd069da5b02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad886cea71c243eda754e50b8e4d8a92'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:48.352254 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-a5c812c8-de17-4323-a7b7-4cd069da5b02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad886cea71c243eda754e50b8e4d8a92'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:48.364147 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-a5c812c8-de17-4323-a7b7-4cd069da5b02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad886cea71c243eda754e50b8e4d8a92'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:48.462775 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-a5c812c8-de17-4323-a7b7-4cd069da5b02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad886cea71c243eda754e50b8e4d8a92'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:48.478202 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1440/2882] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 09:10:48 2022] DELETE /identity/v3/users/ad886cea71c243eda754e50b8e4d8a92 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:10:48.482949 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3e3ad0-bb67-4894-994f-82bc8eefa3ed None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:48.485191 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3e3ad0-bb67-4894-994f-82bc8eefa3ed None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:48.485191 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3e3ad0-bb67-4894-994f-82bc8eefa3ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:48.502487 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-ac3e3ad0-bb67-4894-994f-82bc8eefa3ed None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:10:48.503701 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-ac3e3ad0-bb67-4894-994f-82bc8eefa3ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:48.562792 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1443/2883] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:10:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:10:48.572632 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c812c8-de17-4323-a7b7-4cd069da5b02 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:48.574092 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-a5c812c8-de17-4323-a7b7-4cd069da5b02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:48.632320 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c812c8-de17-4323-a7b7-4cd069da5b02 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:48.635884 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83d7b4d-c6ba-46ca-bd96-fcaa656fa1fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:48.636481 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83d7b4d-c6ba-46ca-bd96-fcaa656fa1fc service neutron] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:48.636801 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83d7b4d-c6ba-46ca-bd96-fcaa656fa1fc service neutron] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:48.637053 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83d7b4d-c6ba-46ca-bd96-fcaa656fa1fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:48.646327 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83d7b4d-c6ba-46ca-bd96-fcaa656fa1fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:48.647774 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83d7b4d-c6ba-46ca-bd96-fcaa656fa1fc service neutron] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:48.656983 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1441/2884] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:10:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:10:49.082832 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3e3ad0-bb67-4894-994f-82bc8eefa3ed None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:49.084329 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-ac3e3ad0-bb67-4894-994f-82bc8eefa3ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:49.096973 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3e3ad0-bb67-4894-994f-82bc8eefa3ed None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:49.101892 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1837dadf-8912-4fdf-898c-421d4600c5a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:49.102513 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1837dadf-8912-4fdf-898c-421d4600c5a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:49.102794 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1837dadf-8912-4fdf-898c-421d4600c5a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:49.103072 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1837dadf-8912-4fdf-898c-421d4600c5a6 admin admin] PATH_INFO: `/v3/projects/5cea599a5b274d17b22d275dc246bb57` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:49.104329 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1837dadf-8912-4fdf-898c-421d4600c5a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5cea599a5b274d17b22d275dc246bb57)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:49.106245 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1837dadf-8912-4fdf-898c-421d4600c5a6 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:49.196615 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-1837dadf-8912-4fdf-898c-421d4600c5a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5cea599a5b274d17b22d275dc246bb57', 'request_id': 'req-1837dadf-8912-4fdf-898c-421d4600c5a6'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:49.202254 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-1837dadf-8912-4fdf-898c-421d4600c5a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5cea599a5b274d17b22d275dc246bb57', 'request_id': 'req-1837dadf-8912-4fdf-898c-421d4600c5a6'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:49.262848 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-1837dadf-8912-4fdf-898c-421d4600c5a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5cea599a5b274d17b22d275dc246bb57', 'request_id': 'req-1837dadf-8912-4fdf-898c-421d4600c5a6'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:49.275063 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-1837dadf-8912-4fdf-898c-421d4600c5a6 admin admin] The token cache is being invalidate because project 5cea599a5b274d17b22d275dc246bb57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:10:49.275351 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-1837dadf-8912-4fdf-898c-421d4600c5a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:49.275981 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-1837dadf-8912-4fdf-898c-421d4600c5a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:10:49.277487 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1444/2885] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 09:10:49 2022] DELETE /identity/v3/projects/5cea599a5b274d17b22d275dc246bb57 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:10:58.652909 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=46,keystone:DELETE=6,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:10:59.271858 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=58,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:11:11.175943 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83d7b4d-c6ba-46ca-bd96-fcaa656fa1fc service neutron] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:11:11.177591 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-f83d7b4d-c6ba-46ca-bd96-fcaa656fa1fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:11:11.191099 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83d7b4d-c6ba-46ca-bd96-fcaa656fa1fc service neutron] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:11:11.247249 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b16f636-4ca4-419f-9212-9c2302db4a86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:11:11.247866 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b16f636-4ca4-419f-9212-9c2302db4a86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:11.248153 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b16f636-4ca4-419f-9212-9c2302db4a86 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:11.248423 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b16f636-4ca4-419f-9212-9c2302db4a86 admin admin] PATH_INFO: `/v3/users/0b9fba36ffcd45a0881a54a82eb704a8` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:11.249578 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b16f636-4ca4-419f-9212-9c2302db4a86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b9fba36ffcd45a0881a54a82eb704a8)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:11:11.251485 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b16f636-4ca4-419f-9212-9c2302db4a86 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:11:11.464793 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-6b16f636-4ca4-419f-9212-9c2302db4a86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b9fba36ffcd45a0881a54a82eb704a8'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:11:11.465584 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-6b16f636-4ca4-419f-9212-9c2302db4a86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b9fba36ffcd45a0881a54a82eb704a8'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:11:11.466215 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-6b16f636-4ca4-419f-9212-9c2302db4a86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b9fba36ffcd45a0881a54a82eb704a8'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:11:11.478490 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-6b16f636-4ca4-419f-9212-9c2302db4a86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b9fba36ffcd45a0881a54a82eb704a8'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:11:11.588436 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-6b16f636-4ca4-419f-9212-9c2302db4a86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b9fba36ffcd45a0881a54a82eb704a8'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:11:11.601769 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1442/2886] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 09:11:11 2022] DELETE /identity/v3/users/0b9fba36ffcd45a0881a54a82eb704a8 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:11:11.606248 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24edb1c1-a830-4fcd-a63a-ef9b78d715cc None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:11.606755 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24edb1c1-a830-4fcd-a63a-ef9b78d715cc None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:11.606882 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24edb1c1-a830-4fcd-a63a-ef9b78d715cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:11.626494 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-24edb1c1-a830-4fcd-a63a-ef9b78d715cc None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:11:11.627833 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-24edb1c1-a830-4fcd-a63a-ef9b78d715cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:11:11.687249 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1445/2887] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:11:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:11:11.697266 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b16f636-4ca4-419f-9212-9c2302db4a86 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:11:11.698786 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-6b16f636-4ca4-419f-9212-9c2302db4a86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:11:11.755574 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b16f636-4ca4-419f-9212-9c2302db4a86 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:11:11.759254 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f042ccf3-b8e6-4227-8970-8db0281c359b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:11:11.759860 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f042ccf3-b8e6-4227-8970-8db0281c359b service neutron] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:11.760140 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f042ccf3-b8e6-4227-8970-8db0281c359b service neutron] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:11.760414 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f042ccf3-b8e6-4227-8970-8db0281c359b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:11.769781 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f042ccf3-b8e6-4227-8970-8db0281c359b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:11:11.771704 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f042ccf3-b8e6-4227-8970-8db0281c359b service neutron] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:11:11.782115 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1443/2888] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:11:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:11:12.230650 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24edb1c1-a830-4fcd-a63a-ef9b78d715cc None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:11:12.232093 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-24edb1c1-a830-4fcd-a63a-ef9b78d715cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:11:12.244793 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24edb1c1-a830-4fcd-a63a-ef9b78d715cc None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:11:12.249826 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4e09fa-5e45-4385-b413-727bbec6cfb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:11:12.250723 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4e09fa-5e45-4385-b413-727bbec6cfb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:12.251011 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4e09fa-5e45-4385-b413-727bbec6cfb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:12.251287 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4e09fa-5e45-4385-b413-727bbec6cfb5 admin admin] PATH_INFO: `/v3/projects/f3af1a6c856340efa227b5925f6c2d7b` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:12.252427 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4e09fa-5e45-4385-b413-727bbec6cfb5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3af1a6c856340efa227b5925f6c2d7b)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:11:12.254377 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4e09fa-5e45-4385-b413-727bbec6cfb5 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:11:12.365498 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-2c4e09fa-5e45-4385-b413-727bbec6cfb5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3af1a6c856340efa227b5925f6c2d7b', 'request_id': 'req-2c4e09fa-5e45-4385-b413-727bbec6cfb5'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:11:12.371087 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-2c4e09fa-5e45-4385-b413-727bbec6cfb5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3af1a6c856340efa227b5925f6c2d7b', 'request_id': 'req-2c4e09fa-5e45-4385-b413-727bbec6cfb5'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:11:12.444792 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-2c4e09fa-5e45-4385-b413-727bbec6cfb5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3af1a6c856340efa227b5925f6c2d7b', 'request_id': 'req-2c4e09fa-5e45-4385-b413-727bbec6cfb5'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:11:12.456122 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-2c4e09fa-5e45-4385-b413-727bbec6cfb5 admin admin] The token cache is being invalidate because project f3af1a6c856340efa227b5925f6c2d7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:11:12.456404 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-2c4e09fa-5e45-4385-b413-727bbec6cfb5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:11:12.457059 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-2c4e09fa-5e45-4385-b413-727bbec6cfb5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:11:12.460029 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1446/2889] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 09:11:12 2022] DELETE /identity/v3/projects/f3af1a6c856340efa227b5925f6c2d7b => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:11:12.463387 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c711de00-3acd-4e94-bf11-da6e77bf5a34 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:12.463869 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c711de00-3acd-4e94-bf11-da6e77bf5a34 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:12.463987 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c711de00-3acd-4e94-bf11-da6e77bf5a34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:12.484428 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-c711de00-3acd-4e94-bf11-da6e77bf5a34 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:11:12.485659 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-c711de00-3acd-4e94-bf11-da6e77bf5a34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:11:12.503584 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:11:12.547336 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1444/2890] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:11:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:11:12.555717 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefcab5d-44b8-457c-9d4b-64bd983018f3 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:12.556001 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefcab5d-44b8-457c-9d4b-64bd983018f3 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:12.556307 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefcab5d-44b8-457c-9d4b-64bd983018f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:12.576234 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-cefcab5d-44b8-457c-9d4b-64bd983018f3 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:11:12.578556 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-cefcab5d-44b8-457c-9d4b-64bd983018f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:11:12.585457 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1447/2891] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:11:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:11:12.589833 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c711de00-3acd-4e94-bf11-da6e77bf5a34 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:11:12.600169 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c711de00-3acd-4e94-bf11-da6e77bf5a34 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:11:12.605000 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb62babb-bc1f-46fa-bc0d-c5acf991f2f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:11:12.605626 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb62babb-bc1f-46fa-bc0d-c5acf991f2f6 admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:12.605903 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb62babb-bc1f-46fa-bc0d-c5acf991f2f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:12.606174 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb62babb-bc1f-46fa-bc0d-c5acf991f2f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:12.606820 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb62babb-bc1f-46fa-bc0d-c5acf991f2f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:11:12.607123 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb62babb-bc1f-46fa-bc0d-c5acf991f2f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:11:12.608858 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb62babb-bc1f-46fa-bc0d-c5acf991f2f6 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:11:12.621032 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1445/2892] 10.222.0.55 () {58 vars in 1153 bytes} [Wed Aug 24 09:11:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:11:12.625348 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefcab5d-44b8-457c-9d4b-64bd983018f3 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:11:12.634518 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefcab5d-44b8-457c-9d4b-64bd983018f3 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:11:12.639129 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9922d55b-f8cf-40de-b8f1-05aea6f5df6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:11:12.639714 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9922d55b-f8cf-40de-b8f1-05aea6f5df6f admin admin] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:12.639986 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9922d55b-f8cf-40de-b8f1-05aea6f5df6f admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:12.640253 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9922d55b-f8cf-40de-b8f1-05aea6f5df6f admin admin] PATH_INFO: `/v3/projects` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:12.640903 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9922d55b-f8cf-40de-b8f1-05aea6f5df6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:11:12.642780 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9922d55b-f8cf-40de-b8f1-05aea6f5df6f admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:11:12.721084 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1448/2893] 10.222.0.55 () {60 vars in 1153 bytes} [Wed Aug 24 09:11:12 2022] POST /identity/v3/projects => generated 397 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:11:12.725813 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb62babb-bc1f-46fa-bc0d-c5acf991f2f6 admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:11:12.735952 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb62babb-bc1f-46fa-bc0d-c5acf991f2f6 admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:11:12.793296 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b70757d-f9eb-421e-b384-9845413dfb3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:11:12.793895 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b70757d-f9eb-421e-b384-9845413dfb3a admin admin] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:12.794176 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b70757d-f9eb-421e-b384-9845413dfb3a admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:12.794469 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b70757d-f9eb-421e-b384-9845413dfb3a admin admin] PATH_INFO: `/v3/users` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:12.795106 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b70757d-f9eb-421e-b384-9845413dfb3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:11:12.797076 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b70757d-f9eb-421e-b384-9845413dfb3a admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:11:12.913802 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1446/2894] 10.222.0.55 () {60 vars in 1144 bytes} [Wed Aug 24 09:11:12 2022] POST /identity/v3/users => generated 299 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:11:12.917893 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9922d55b-f8cf-40de-b8f1-05aea6f5df6f admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:11:12.928022 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9922d55b-f8cf-40de-b8f1-05aea6f5df6f admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:11:12.932558 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9597d9-3ccf-4d45-9510-a5f7834d574e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:11:12.933147 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9597d9-3ccf-4d45-9510-a5f7834d574e admin admin] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:12.933421 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9597d9-3ccf-4d45-9510-a5f7834d574e admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:12.933691 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9597d9-3ccf-4d45-9510-a5f7834d574e admin admin] PATH_INFO: `/v3/roles` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:12.934319 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9597d9-3ccf-4d45-9510-a5f7834d574e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:11:12.935943 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9597d9-3ccf-4d45-9510-a5f7834d574e admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:11:12.946303 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1449/2895] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:11:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:11:12.950530 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b70757d-f9eb-421e-b384-9845413dfb3a admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:11:12.961550 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b70757d-f9eb-421e-b384-9845413dfb3a admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:11:12.966209 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f4a6a2-754e-4b28-a338-c1cb47a49fbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:11:12.966798 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f4a6a2-754e-4b28-a338-c1cb47a49fbb admin admin] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:12.967076 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f4a6a2-754e-4b28-a338-c1cb47a49fbb admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:12.967347 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f4a6a2-754e-4b28-a338-c1cb47a49fbb admin admin] PATH_INFO: `/v3/roles` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:12.967974 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f4a6a2-754e-4b28-a338-c1cb47a49fbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:11:12.969585 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f4a6a2-754e-4b28-a338-c1cb47a49fbb admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:11:12.979965 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1447/2896] 10.222.0.55 () {58 vars in 1122 bytes} [Wed Aug 24 09:11:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:11:12.984067 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9597d9-3ccf-4d45-9510-a5f7834d574e admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:11:12.994713 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9597d9-3ccf-4d45-9510-a5f7834d574e admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:11:12.999257 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9becb60-ea83-4336-8c78-6483be4ffee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:11:12.999898 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9becb60-ea83-4336-8c78-6483be4ffee6 admin admin] REQUEST_METHOD: `PUT` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:13.000177 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9becb60-ea83-4336-8c78-6483be4ffee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:13.000449 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9becb60-ea83-4336-8c78-6483be4ffee6 admin admin] PATH_INFO: `/v3/projects/1e93464da4fd489e98b94a8dca6b2409/users/b0aa0ddc93844511a0b803988c442d83/roles/b674fd0a592e4e0cada811bba5ceea4d` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:13.015963 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9becb60-ea83-4336-8c78-6483be4ffee6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e93464da4fd489e98b94a8dca6b2409, user_id=b0aa0ddc93844511a0b803988c442d83, role_id=b674fd0a592e4e0cada811bba5ceea4d)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:11:13.018634 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9becb60-ea83-4336-8c78-6483be4ffee6 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:11:13.085690 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1450/2897] 10.222.0.55 () {60 vars in 1483 bytes} [Wed Aug 24 09:11:12 2022] PUT /identity/v3/projects/1e93464da4fd489e98b94a8dca6b2409/users/b0aa0ddc93844511a0b803988c442d83/roles/b674fd0a592e4e0cada811bba5ceea4d => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:11:13.097247 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad7ffbf-8ad3-427f-8034-d2a50a621b25 None None] REQUEST_METHOD: `POST` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:13.097528 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad7ffbf-8ad3-427f-8034-d2a50a621b25 None None] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:13.097802 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad7ffbf-8ad3-427f-8034-d2a50a621b25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:13.125291 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.auth.core [None req-3ad7ffbf-8ad3-427f-8034-d2a50a621b25 None None] MFA Rules not processed for user `b0aa0ddc93844511a0b803988c442d83`. Rule list: `[]` (Enabled: `True`). {{(pid=93542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:11:13.126535 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-3ad7ffbf-8ad3-427f-8034-d2a50a621b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:11:13.190684 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1448/2898] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:11:13 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:11:13.204794 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9becb60-ea83-4336-8c78-6483be4ffee6 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:11:13.206505 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-c9becb60-ea83-4336-8c78-6483be4ffee6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:11:13.227753 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:11:13.262461 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9becb60-ea83-4336-8c78-6483be4ffee6 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:11:13.266369 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4994556-4fa1-455e-b0b5-092d9dcd1f32 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbb9ac667f04c35a63b2689df18efed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:11:13.266968 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4994556-4fa1-455e-b0b5-092d9dcd1f32 service cinder] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:13.267271 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4994556-4fa1-455e-b0b5-092d9dcd1f32 service cinder] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:13.267542 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4994556-4fa1-455e-b0b5-092d9dcd1f32 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:13.276132 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4994556-4fa1-455e-b0b5-092d9dcd1f32 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:11:13.277606 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4994556-4fa1-455e-b0b5-092d9dcd1f32 service cinder] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:11:13.287074 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1451/2899] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:11:13 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:11:23.187840 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:11:23.225538 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad7ffbf-8ad3-427f-8034-d2a50a621b25 None None] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:11:23.288469 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad7ffbf-8ad3-427f-8034-d2a50a621b25 None None] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:11:23.293072 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d305cd55-b07b-4eb0-9e8f-b82b172936bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:11:23.293697 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d305cd55-b07b-4eb0-9e8f-b82b172936bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:23.293974 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d305cd55-b07b-4eb0-9e8f-b82b172936bc admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:23.294252 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d305cd55-b07b-4eb0-9e8f-b82b172936bc admin admin] PATH_INFO: `/v3/users/b0aa0ddc93844511a0b803988c442d83` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:23.295368 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d305cd55-b07b-4eb0-9e8f-b82b172936bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0aa0ddc93844511a0b803988c442d83)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:11:23.297324 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d305cd55-b07b-4eb0-9e8f-b82b172936bc admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:11:23.342503 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=17 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:11:23.414765 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-d305cd55-b07b-4eb0-9e8f-b82b172936bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0aa0ddc93844511a0b803988c442d83'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:11:23.415608 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-d305cd55-b07b-4eb0-9e8f-b82b172936bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0aa0ddc93844511a0b803988c442d83'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:11:23.416348 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-d305cd55-b07b-4eb0-9e8f-b82b172936bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0aa0ddc93844511a0b803988c442d83'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:11:23.429047 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-d305cd55-b07b-4eb0-9e8f-b82b172936bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0aa0ddc93844511a0b803988c442d83'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:11:23.479564 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-d305cd55-b07b-4eb0-9e8f-b82b172936bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0aa0ddc93844511a0b803988c442d83'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:11:23.491960 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1449/2900] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 09:11:23 2022] DELETE /identity/v3/users/b0aa0ddc93844511a0b803988c442d83 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:11:23.496783 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca346f3-83ae-4c63-a9ca-4d4119375122 None None] REQUEST_METHOD: `POST` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:23.503805 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca346f3-83ae-4c63-a9ca-4d4119375122 None None] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:23.503805 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca346f3-83ae-4c63-a9ca-4d4119375122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:23.516248 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.auth.core [None req-4ca346f3-83ae-4c63-a9ca-4d4119375122 None None] MFA Rules not processed for user `28800687066c4de989fe1940c9a30412`. Rule list: `[]` (Enabled: `True`). {{(pid=93543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:11:23.517571 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-4ca346f3-83ae-4c63-a9ca-4d4119375122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:11:23.577172 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1452/2901] 10.222.0.55 () {58 vars in 958 bytes} [Wed Aug 24 09:11:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:11:23.587502 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d305cd55-b07b-4eb0-9e8f-b82b172936bc admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:11:23.589021 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-d305cd55-b07b-4eb0-9e8f-b82b172936bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:11:23.647276 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d305cd55-b07b-4eb0-9e8f-b82b172936bc admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:11:23.650929 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89551a9-6ee8-4e31-93b9-9de8fe31ad60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:11:23.651535 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89551a9-6ee8-4e31-93b9-9de8fe31ad60 service neutron] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:23.651814 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89551a9-6ee8-4e31-93b9-9de8fe31ad60 service neutron] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:23.652089 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89551a9-6ee8-4e31-93b9-9de8fe31ad60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:23.660967 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89551a9-6ee8-4e31-93b9-9de8fe31ad60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:11:23.662957 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89551a9-6ee8-4e31-93b9-9de8fe31ad60 service neutron] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:11:23.672678 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1450/2902] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:11:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:11:24.047208 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca346f3-83ae-4c63-a9ca-4d4119375122 None None] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:11:24.053055 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-4ca346f3-83ae-4c63-a9ca-4d4119375122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:11:24.062618 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca346f3-83ae-4c63-a9ca-4d4119375122 None None] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:11:24.067282 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532670ae-b147-41a8-9663-65824e945de7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:11:24.068406 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532670ae-b147-41a8-9663-65824e945de7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:24.068723 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532670ae-b147-41a8-9663-65824e945de7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:24.069465 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532670ae-b147-41a8-9663-65824e945de7 admin admin] PATH_INFO: `/v3/projects/1e93464da4fd489e98b94a8dca6b2409` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:24.070601 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532670ae-b147-41a8-9663-65824e945de7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e93464da4fd489e98b94a8dca6b2409)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:11:24.073181 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532670ae-b147-41a8-9663-65824e945de7 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:11:24.209665 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-532670ae-b147-41a8-9663-65824e945de7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e93464da4fd489e98b94a8dca6b2409', 'request_id': 'req-532670ae-b147-41a8-9663-65824e945de7'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:11:24.221905 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-532670ae-b147-41a8-9663-65824e945de7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e93464da4fd489e98b94a8dca6b2409', 'request_id': 'req-532670ae-b147-41a8-9663-65824e945de7'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:11:24.274019 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-532670ae-b147-41a8-9663-65824e945de7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e93464da4fd489e98b94a8dca6b2409', 'request_id': 'req-532670ae-b147-41a8-9663-65824e945de7'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:11:24.285460 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-532670ae-b147-41a8-9663-65824e945de7 admin admin] The token cache is being invalidate because project 1e93464da4fd489e98b94a8dca6b2409 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:11:24.285825 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-532670ae-b147-41a8-9663-65824e945de7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:11:24.286247 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-532670ae-b147-41a8-9663-65824e945de7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:11:24.287740 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1453/2903] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 09:11:24 2022] DELETE /identity/v3/projects/1e93464da4fd489e98b94a8dca6b2409 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:11:33.668413 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:11:34.282391 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:12:10.893697 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89551a9-6ee8-4e31-93b9-9de8fe31ad60 service neutron] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:12:10.903089 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89551a9-6ee8-4e31-93b9-9de8fe31ad60 service neutron] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:12:10.950955 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abb42cd-cb77-4df7-9746-c19fdf6c4912 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:12:10.951551 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abb42cd-cb77-4df7-9746-c19fdf6c4912 service neutron] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:12:10.951831 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abb42cd-cb77-4df7-9746-c19fdf6c4912 service neutron] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:12:10.952105 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abb42cd-cb77-4df7-9746-c19fdf6c4912 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:12:10.953856 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-0abb42cd-cb77-4df7-9746-c19fdf6c4912 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:12:11.011020 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abb42cd-cb77-4df7-9746-c19fdf6c4912 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:12:11.012993 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abb42cd-cb77-4df7-9746-c19fdf6c4912 service neutron] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:12:11.022015 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1451/2904] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:12:10 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:12:11.915080 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532670ae-b147-41a8-9663-65824e945de7 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:12:11.915080 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-532670ae-b147-41a8-9663-65824e945de7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:12:11.996014 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532670ae-b147-41a8-9663-65824e945de7 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:12:12.002409 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728d3743-8779-4afe-9734-c9d0f533564a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17da357e555420799ee681bffbcc268', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:12:12.003029 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728d3743-8779-4afe-9734-c9d0f533564a service nova] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:12:12.003311 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728d3743-8779-4afe-9734-c9d0f533564a service nova] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:12:12.003584 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728d3743-8779-4afe-9734-c9d0f533564a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:12:12.005377 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-728d3743-8779-4afe-9734-c9d0f533564a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:12:12.018301 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728d3743-8779-4afe-9734-c9d0f533564a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:12:12.019900 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728d3743-8779-4afe-9734-c9d0f533564a service nova] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:12:12.032595 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1454/2905] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:12:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:12:12.131204 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abb42cd-cb77-4df7-9746-c19fdf6c4912 service neutron] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:12:12.132976 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-0abb42cd-cb77-4df7-9746-c19fdf6c4912 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:12:12.144786 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abb42cd-cb77-4df7-9746-c19fdf6c4912 service neutron] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:12:12.148873 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d883c2b-579a-449c-ad73-523cfceab5f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:12:12.149503 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d883c2b-579a-449c-ad73-523cfceab5f6 service neutron] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:12:12.149782 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d883c2b-579a-449c-ad73-523cfceab5f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:12:12.150056 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d883c2b-579a-449c-ad73-523cfceab5f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:12:12.152007 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-4d883c2b-579a-449c-ad73-523cfceab5f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:12:12.163810 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d883c2b-579a-449c-ad73-523cfceab5f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:12:12.165529 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d883c2b-579a-449c-ad73-523cfceab5f6 service neutron] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:12:12.175806 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1452/2906] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:12:12 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:12:12.274768 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728d3743-8779-4afe-9734-c9d0f533564a service nova] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:12:12.276177 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-728d3743-8779-4afe-9734-c9d0f533564a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:12:12.288315 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728d3743-8779-4afe-9734-c9d0f533564a service nova] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:12:12.292425 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8305f8e7-bfb5-467c-9f19-93c743e292f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:12:12.293632 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8305f8e7-bfb5-467c-9f19-93c743e292f2 service neutron] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:12:12.293889 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8305f8e7-bfb5-467c-9f19-93c743e292f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:12:12.294166 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8305f8e7-bfb5-467c-9f19-93c743e292f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:12:12.296367 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-8305f8e7-bfb5-467c-9f19-93c743e292f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:12:12.310560 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8305f8e7-bfb5-467c-9f19-93c743e292f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:12:12.312028 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8305f8e7-bfb5-467c-9f19-93c743e292f2 service neutron] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:12:12.324411 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1455/2907] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:12:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:12:13.511246 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d883c2b-579a-449c-ad73-523cfceab5f6 service neutron] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:12:13.520262 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d883c2b-579a-449c-ad73-523cfceab5f6 service neutron] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:12:13.523943 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2392b910-12fb-4cbb-8cbe-fc6427ef48e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:12:13.524561 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2392b910-12fb-4cbb-8cbe-fc6427ef48e1 service neutron] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:12:13.524831 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2392b910-12fb-4cbb-8cbe-fc6427ef48e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:12:13.525099 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2392b910-12fb-4cbb-8cbe-fc6427ef48e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:12:13.526824 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-2392b910-12fb-4cbb-8cbe-fc6427ef48e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:12:13.542546 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2392b910-12fb-4cbb-8cbe-fc6427ef48e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:12:13.544530 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2392b910-12fb-4cbb-8cbe-fc6427ef48e1 service neutron] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:12:13.556455 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1453/2908] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:12:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:12:17.330102 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8305f8e7-bfb5-467c-9f19-93c743e292f2 service neutron] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:12:17.339036 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8305f8e7-bfb5-467c-9f19-93c743e292f2 service neutron] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:12:17.342793 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a335863-727c-4932-acce-1196bf94c179 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28b58388d1664510b0b507323bca5ae8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:12:17.348600 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a335863-727c-4932-acce-1196bf94c179 service neutron] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:12:17.348600 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a335863-727c-4932-acce-1196bf94c179 service neutron] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:12:17.348600 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a335863-727c-4932-acce-1196bf94c179 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:12:17.348600 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-9a335863-727c-4932-acce-1196bf94c179 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:12:17.408031 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a335863-727c-4932-acce-1196bf94c179 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:12:17.409495 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a335863-727c-4932-acce-1196bf94c179 service neutron] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:12:17.419563 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1456/2909] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:12:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:12:20.841640 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2392b910-12fb-4cbb-8cbe-fc6427ef48e1 service neutron] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:12:20.848703 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-2392b910-12fb-4cbb-8cbe-fc6427ef48e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:12:20.914738 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2392b910-12fb-4cbb-8cbe-fc6427ef48e1 service neutron] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:12:20.920259 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e365f9-50dd-45dd-a59c-afd96e1e48b0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf0c99771661413aaa0f237af6b394ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:12:20.920905 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e365f9-50dd-45dd-a59c-afd96e1e48b0 service placement] REQUEST_METHOD: `GET` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:12:20.921182 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e365f9-50dd-45dd-a59c-afd96e1e48b0 service placement] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:12:20.921454 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e365f9-50dd-45dd-a59c-afd96e1e48b0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:12:20.923196 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-c8e365f9-50dd-45dd-a59c-afd96e1e48b0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:12:20.935131 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e365f9-50dd-45dd-a59c-afd96e1e48b0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:12:20.936725 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e365f9-50dd-45dd-a59c-afd96e1e48b0 service placement] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:12:20.946923 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1454/2910] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:12:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:12:24.587583 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a335863-727c-4932-acce-1196bf94c179 service neutron] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:12:24.591891 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-9a335863-727c-4932-acce-1196bf94c179 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:12:24.602662 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a335863-727c-4932-acce-1196bf94c179 service neutron] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:12:24.607176 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23195f76-3452-433e-8822-a039bb4f864b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:12:24.607803 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23195f76-3452-433e-8822-a039bb4f864b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:12:24.608105 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23195f76-3452-433e-8822-a039bb4f864b admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:12:24.608398 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23195f76-3452-433e-8822-a039bb4f864b admin admin] PATH_INFO: `/v3/users/faf0d3eebf714e8891c6a3eebf8d87a9` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:12:24.609555 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23195f76-3452-433e-8822-a039bb4f864b admin admin] RBAC: Authorizing `identity:delete_user(user_id=faf0d3eebf714e8891c6a3eebf8d87a9)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:12:24.611447 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23195f76-3452-433e-8822-a039bb4f864b admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:12:24.791152 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-23195f76-3452-433e-8822-a039bb4f864b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'faf0d3eebf714e8891c6a3eebf8d87a9'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:12:24.791799 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-23195f76-3452-433e-8822-a039bb4f864b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'faf0d3eebf714e8891c6a3eebf8d87a9'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:12:24.792475 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-23195f76-3452-433e-8822-a039bb4f864b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'faf0d3eebf714e8891c6a3eebf8d87a9'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:12:24.803299 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:12:24.805311 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-23195f76-3452-433e-8822-a039bb4f864b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'faf0d3eebf714e8891c6a3eebf8d87a9'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:12:24.890964 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-23195f76-3452-433e-8822-a039bb4f864b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'faf0d3eebf714e8891c6a3eebf8d87a9'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:12:24.903508 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1457/2911] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 09:12:24 2022] DELETE /identity/v3/users/faf0d3eebf714e8891c6a3eebf8d87a9 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:12:25.168757 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e365f9-50dd-45dd-a59c-afd96e1e48b0 service placement] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:12:25.181839 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e365f9-50dd-45dd-a59c-afd96e1e48b0 service placement] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:12:25.205039 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=100 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:12:25.253166 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea838f63-61d4-4f41-96ee-47dc39cb3ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:12:25.253807 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea838f63-61d4-4f41-96ee-47dc39cb3ebe admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:12:25.254086 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea838f63-61d4-4f41-96ee-47dc39cb3ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:12:25.254381 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea838f63-61d4-4f41-96ee-47dc39cb3ebe admin admin] PATH_INFO: `/v3/projects/e19c750fc9af412f93ff6369e4c57730` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:12:25.255547 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea838f63-61d4-4f41-96ee-47dc39cb3ebe admin admin] RBAC: Authorizing `identity:delete_project(project_id=e19c750fc9af412f93ff6369e4c57730)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:12:25.257554 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea838f63-61d4-4f41-96ee-47dc39cb3ebe admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:12:25.389202 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-ea838f63-61d4-4f41-96ee-47dc39cb3ebe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e19c750fc9af412f93ff6369e4c57730', 'request_id': 'req-ea838f63-61d4-4f41-96ee-47dc39cb3ebe'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:12:25.479536 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-ea838f63-61d4-4f41-96ee-47dc39cb3ebe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e19c750fc9af412f93ff6369e4c57730', 'request_id': 'req-ea838f63-61d4-4f41-96ee-47dc39cb3ebe'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:12:25.486688 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-ea838f63-61d4-4f41-96ee-47dc39cb3ebe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e19c750fc9af412f93ff6369e4c57730', 'request_id': 'req-ea838f63-61d4-4f41-96ee-47dc39cb3ebe'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:12:25.498343 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-ea838f63-61d4-4f41-96ee-47dc39cb3ebe admin admin] The token cache is being invalidate because project e19c750fc9af412f93ff6369e4c57730 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:12:25.498675 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-ea838f63-61d4-4f41-96ee-47dc39cb3ebe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:12:25.499381 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-ea838f63-61d4-4f41-96ee-47dc39cb3ebe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:12:25.500939 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1455/2912] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 09:12:25 2022] DELETE /identity/v3/projects/e19c750fc9af412f93ff6369e4c57730 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:12:25.505754 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23195f76-3452-433e-8822-a039bb4f864b admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:12:25.506342 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-23195f76-3452-433e-8822-a039bb4f864b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:12:25.519454 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23195f76-3452-433e-8822-a039bb4f864b admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:12:25.575569 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c146c113-f7d2-47f1-bde6-56558a64e6d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:12:25.576190 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c146c113-f7d2-47f1-bde6-56558a64e6d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:12:25.576464 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c146c113-f7d2-47f1-bde6-56558a64e6d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:12:25.576764 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c146c113-f7d2-47f1-bde6-56558a64e6d3 admin admin] PATH_INFO: `/v3/users/0afee2bcfef34b6bb64d0bfe4dc7dab9` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:12:25.577874 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c146c113-f7d2-47f1-bde6-56558a64e6d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0afee2bcfef34b6bb64d0bfe4dc7dab9)` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:12:25.579778 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c146c113-f7d2-47f1-bde6-56558a64e6d3 admin admin] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:12:25.760991 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-c146c113-f7d2-47f1-bde6-56558a64e6d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0afee2bcfef34b6bb64d0bfe4dc7dab9'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:12:25.761637 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-c146c113-f7d2-47f1-bde6-56558a64e6d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0afee2bcfef34b6bb64d0bfe4dc7dab9'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:12:25.762149 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-c146c113-f7d2-47f1-bde6-56558a64e6d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0afee2bcfef34b6bb64d0bfe4dc7dab9'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:12:25.773727 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-c146c113-f7d2-47f1-bde6-56558a64e6d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0afee2bcfef34b6bb64d0bfe4dc7dab9'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:12:25.850774 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.notifications [None req-c146c113-f7d2-47f1-bde6-56558a64e6d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0afee2bcfef34b6bb64d0bfe4dc7dab9'} {{(pid=93543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:12:25.864913 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1458/2913] 10.222.0.55 () {58 vars in 1224 bytes} [Wed Aug 24 09:12:25 2022] DELETE /identity/v3/users/0afee2bcfef34b6bb64d0bfe4dc7dab9 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:12:26.146537 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea838f63-61d4-4f41-96ee-47dc39cb3ebe admin admin] Authenticating user token {{(pid=93542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:12:26.146537 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.fernet_utils [None req-ea838f63-61d4-4f41-96ee-47dc39cb3ebe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:12:26.164623 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea838f63-61d4-4f41-96ee-47dc39cb3ebe admin admin] Validating token access rules against request {{(pid=93542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:12:26.227157 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a631020-216c-4f88-ab9b-10fef539cc95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28800687066c4de989fe1940c9a30412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ad87fa1813247aba83e93f886f34eb3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:12:26.227900 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a631020-216c-4f88-ab9b-10fef539cc95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:12:26.228201 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a631020-216c-4f88-ab9b-10fef539cc95 admin admin] SCRIPT_NAME: `/identity` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:12:26.228456 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a631020-216c-4f88-ab9b-10fef539cc95 admin admin] PATH_INFO: `/v3/projects/d8ad005a17d043ecbbab5cf0ecc11ac8` {{(pid=93542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:12:26.229615 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a631020-216c-4f88-ab9b-10fef539cc95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8ad005a17d043ecbbab5cf0ecc11ac8)` {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:12:26.231510 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a631020-216c-4f88-ab9b-10fef539cc95 admin admin] RBAC: Authorization granted {{(pid=93542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:12:26.377371 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-9a631020-216c-4f88-ab9b-10fef539cc95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8ad005a17d043ecbbab5cf0ecc11ac8', 'request_id': 'req-9a631020-216c-4f88-ab9b-10fef539cc95'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:12:26.378967 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:12:26.462062 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-9a631020-216c-4f88-ab9b-10fef539cc95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8ad005a17d043ecbbab5cf0ecc11ac8', 'request_id': 'req-9a631020-216c-4f88-ab9b-10fef539cc95'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:12:26.468771 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-9a631020-216c-4f88-ab9b-10fef539cc95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8ad005a17d043ecbbab5cf0ecc11ac8', 'request_id': 'req-9a631020-216c-4f88-ab9b-10fef539cc95'} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:12:26.478964 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-9a631020-216c-4f88-ab9b-10fef539cc95 admin admin] The token cache is being invalidate because project d8ad005a17d043ecbbab5cf0ecc11ac8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:12:26.479249 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-9a631020-216c-4f88-ab9b-10fef539cc95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:12:26.479878 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG keystone.notifications [None req-9a631020-216c-4f88-ab9b-10fef539cc95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:12:26.481369 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: [pid: 93542|app: 0|req: 1456/2914] 10.222.0.55 () {58 vars in 1233 bytes} [Wed Aug 24 09:12:26 2022] DELETE /identity/v3/projects/d8ad005a17d043ecbbab5cf0ecc11ac8 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:12:35.860944 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:DELETE=12,keystone:SELECT=60,keystone:INSERT=2 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:12:36.662524 dv-cinder-852210-4-iscsi devstack@keystone.service[93542]: DEBUG dbcounter [-] [93542] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=6 {{(pid=93542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:14:44.572645 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c146c113-f7d2-47f1-bde6-56558a64e6d3 admin admin] Authenticating user token {{(pid=93543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:14:44.574075 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-c146c113-f7d2-47f1-bde6-56558a64e6d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:14:44.713963 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c146c113-f7d2-47f1-bde6-56558a64e6d3 admin admin] Validating token access rules against request {{(pid=93543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:14:44.771933 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d8779f-0f65-4443-9ae8-381677693317 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf0c99771661413aaa0f237af6b394ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '420b18725f2342159646fb86438318fd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:14:44.772579 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d8779f-0f65-4443-9ae8-381677693317 service placement] REQUEST_METHOD: `GET` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:14:44.773501 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d8779f-0f65-4443-9ae8-381677693317 service placement] SCRIPT_NAME: `/identity` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:14:44.773790 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d8779f-0f65-4443-9ae8-381677693317 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:14:44.775552 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.fernet_utils [None req-82d8779f-0f65-4443-9ae8-381677693317 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:14:44.786898 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d8779f-0f65-4443-9ae8-381677693317 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:14:44.788483 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d8779f-0f65-4443-9ae8-381677693317 service placement] RBAC: Authorization granted {{(pid=93543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:14:44.798735 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: [pid: 93543|app: 0|req: 1459/2915] 10.222.0.55 () {60 vars in 1365 bytes} [Wed Aug 24 09:14:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:14:54.794402 dv-cinder-852210-4-iscsi devstack@keystone.service[93543]: DEBUG dbcounter [-] [93543] Writing DB stats keystone:SELECT=89 {{(pid=93543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}